Evaluation of Software Systems - Report Example

INFORMATION SYSTEMS REVIEW AND REPORT STUDENT NAME PROFFESSOR AFFLIATE INSTITUTION DATE Table of Contents 1.0 Introduction 4 1.2 Background 4 1.3 Objectives 5 1.4 Scope 5 1.5 Evaluation criteria 6 2.0 System evaluation 6 2.1 www.131500.com.au 6 10 Process analysis 11 Process analysis 13 yes 13 Usability analysis 14 Rating system 14 Virgin australia system 15 Process Analysis 17 20 Process diagram 21 Usability analysis 22 Rating system 24 Strength 24 Weakness 24 University of Technology Sydney (http://www.lib.uts.edu.au) 25 Features of the system 27 Process analysis 29 30 31 Usability analysis 31 Rating system 32 Strength 32 Recommendation 33 Conclusion 34 References 34 Appendices 35 Usability checklist 35 Process evaluation checklist 36 1.0 Introduction System evaluation is a critical process that enables an organization to evaluate and determine the usefulness of a system. The results of the evaluation is a report that would be used by the stalk holders to determine whether the system meets the user and the organizational requirements and whether the system can be retained or replaced with a new and more efficient system. System review could also be done on an existing system during system acquisition to select a suitable system that satisfies the organization’s missions. This is a report of a review of three web-based software systems (Virgin Australia airline system, UTS web-based library System and New South Wales state public transport information system) to evaluate and determine the most useful and salient features in a web-based software system to be integrated in the new company’s website that is to replace the current outdated and non-interactive website. The review was carried out on 20th March 2013 and was authorized by the company’s CEO Mr Carson Joshua. 1.2 Background The current company’s website does not meet the company’s objectives as originally, it was just intended to provide the company with a basic web presence (and reserve the domain name), and so lack of function was not such a big issue at that time. However, with the growth in internet potential for improving retail image, online sales and lowering overheads, the company’s management now wishes to use the website as a more user-friendly means for consumer’s to purchase goods direct from the company. The existing design is quite old and unattractive, providing only routine information without much user interaction and system upgrade would not be viable to the company. The management has authorized for an evaluation of current web-based information systems to try and determine what design advantages might also usefully be considered for the organization’s web based system redevelopment. 1.3 Objectives The review was guided by the following objectives: 1. To compare alternative software to determine the best alternative that best fit the organization’s need 2. To determine desirable qualities that can be integrate in the company’s website during redevelopment phase. 1.4 Scope The review was confined to three systems www.131500.com.au www.virginaustralia.com/نسخة مخبأة http://www.lib.uts.edu.au/find 1.5 Evaluation criteria The concept of usability was used as an evaluation criterion. According to ISO 9241 Usability of a product is the degree to which the product can be used by specific users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specific context of use. The review would be evaluating the usefulness of the system. The system usability can be determined based on the three aspects of usability, which are effectiveness, efficiency and satisfaction. The system usability was evaluated by exposing the system to users and analyzing the time the user take to interact with the system, the accuracy of the data and the adequacy of data captured. 2.0 System evaluation 2.1 www.131500.com.au The web based system is for New South Wales state public transport information system. the website is an information site that provide the user with all the transport information pertaining to the New South Wales state. In the website’s home page multi-layouts has been applied to the page but the major layout is the three column layout. The right hand side of the home page is the featured events while the middle column is a form that a user can use to schedule and get trip/ travel information. The left hand side of the page is the main links that enable the users to access the whole system. This menus includes Plan your trip links to tools and information that help users plan their trips Tickets give the user ticketing information Maps privide the users with maps on the various transport network Events give a user a general information of the upcoming events. Attractions give a general information on the various attraction sites within new south wales state Customer service provive the user with the contact information for feedback, public amenities and who do contact incase of lost items. About Us provide the information on the system and the entire transport system in new south wales. FAQS is link to the most frequently asked questions. This can assist users to find help or more information when confronted with difficulties in carrying out a given task in the system Figure 1: Home page for the NSW transport information website The webpage is laid down in a way that importance links are categorized based on the group of services the page is intended to display to the users. This method of categorizing related functionalities into one control enables proper organization of the page. The consistent page layout ensures smooth navigation through the website. Figure 2: Website layout map Figure 3: General Website Layout Process analysis The selected task is the trip planning for a new trip. The user is supposed to click the Plan Your Trip link to be able to fill his/her trip preference and submit the information Figure 4: New trip plan form Once the user clicks the GO button the user is directed to a confirmation page where he can decide to proceed to the next step or edit the preferences. Figure 5: If the user clicks continue button he proceeds to the results page Figure 6: Trip planner results Process analysis Amend trip yes Figure 7: Process analysis Usability analysis Based on the usability checklist the following finding was found. The website is purely an informational website. 1. Learnability The system is easy to use and the users do not require much time to get familiarize themselves to the system. The contents are categorized appropriately 2. Memorability Since the main menus are categorized based on the task classification the users could easily remember the routes taken to achieve a give task. The routes to achieving a given task are also minimum and average of 3 links. 3. Efficiency and effectiveness The system is both efficient and effective as it is able to achieve the expected goal of effectively managing the library resources 4. Error proneness The system is not prone to errors because it has been tested 5. Consistency The page layout and the web pages are consistent 6. Help System Below each link are the explanations on its functionality. A help link is available for users who might get confused on the way. Rating system On a scale of 1 to 5 rate the system, where 1 is the worst and 5 is the highest score. USABILITY ASPECT SCORE Learnability 3 Memorability 3 Efficiency 4 Effectiveness 4 Error proneness 4 Consistency 5 Help System 4 TOTAL SCORE 27 Table 1: Rating System Virgin australia system The virgin Australia system is flight information and booking systems. The website provides information related to the virgin atlantics lights. Figure 8: Home page The controls in the website are arranged in horizontal tabs. Each tab has a number of related information that appear when the mouse curse hover over the tab. Related services that appear when the cursor hover are grouped into categories and under each category are sub-categories. The sub-categories are links to the information pages which provide detailed information on the stated title. Figure 9: Website layout diagram Figure 10: Snapshot of the website layout The home page in figure 8 has main tabs that allow users to access all the website resources. The main tabs are: Experience: give a user links to information on various experiences the user expect to have in the airport, airplane, service experience as offered by the airline Planning: provide further links for user information on flight booking, flight fares, baggage charges, international travel among other information a user might require in preparation for a trip. Booking : general information on booking a services, flight, hotels, parking space among others Special offers: provide the user with the latest offers offered by the airline. Velocity frequent flyer: general information on the velocity frequent flyer To test for usability was done by setting some system scenarios. In this evaluation we used flight booking process. Process Analysis For the customer to book a flight in Virgin Australia airline he/she would have to enter the following web address: http://www.virginaustralia.com/au. Once the home page has loaded the user should go to the booking tap and under the flight section select the book flight link. The link would direct the user to a web booking form. Figure 11: Booking tab The user is expected to fill the booking form: the origin, destination, departure, return, age and travel class. Figure 12: Flight booking form If the flight is not available at the particular selected date the user is prompted to amend the information and resubmit. Figure 13: information amendment form If the flight is available the results are displayed, showing the various flight charges. Once the flight availability is confirmed the passenger is expected to fill personal details, then extra details then select the seat position and pay the flight charges. Process diagram No Usability analysis Based on the usability checklist the following finding was found. 7. Learnability The contents are categorized appropriately but the website links are linked to so many external website that make it difficult to learn. The routes to achieving a given task are also more and average of 4 links. 8. Memorability Since the main menus are categorized based on the task classification the users could easily remember the routes taken to achieve a give task except few instances where the web directs the user to an external website with completely different page. The routes to achieving a given task are also minimum and average of 3 links. 9. Efficiency and effectiveness The system is both efficient and effective as it is able to achieve the expected goal of effectively managing the flight information 10. Error proneness The system is not prone to errors because it has been tested 11. Consistency The page layout and the web pages are consistent 12. Help System The system has a lot of information on how to achieve a given task in separate pages and also there exist a virtual chatting tool for user to ask questions. Rating system On a scale of 1 to 5 rate the system, where 1 is the worst and 5 is the highest score. USABILITY ASPECT SCORE Learnability 2 Memorability 3 Efficiency 4 Effectiveness 4 Error proneness 4 Consistency 4 Help System 3 TOTAL SCORE 22 Strength The use of tabs is good feature as it allows the user to access information without having to move from one page to another. The arrangement of items into related task makes it easy for the user to learn and memorize the system. Weakness The system has numerous links to external website thus creating the interface inconsistency and confusion to the part of a naïve user The system is relative slow When booking for a flight the system should show the available flights between a given time period to avoid the customer having to guess the availability University of Technology Sydney (http://www.lib.uts.edu.au) The University of Technology Sydney (UTS) library system provides information to faculty, students, and others about the libraries’ functions and research material. The web-based library management system is capable of managing all the activities in the library and providing useful information to users of the library. For example the website provide the users with powerful tools for searching any library resources. Such library resources include the articles, eBooks, references materials. The find function can search upto the location where the resource is. The search can be customized based on the user knowledge of the system to filter the results to specific hit. The system can provide the information on the spaces in the library for example it can display whole of the library floors layout. The system also provide information on the specialized places suited to the individual convinience for exaple quiet study rooms etc. The system can be used to issue books, return borrowed books, renew a book and provide information on how to loan a book and how to pay overdue charges. The system further provide a number of research tools to research, teaching/learning materials for both teachers and students, links to the publications published by the university, information on various upcoming events and various information pertaining to the college staff and infractructure. On the first page of the UTS Libraries’ website there are various links that will take a user deeper into the site. The quick links are located on the right hand side of the homepage and gives advance users shortcuts to achieving a specific task such as find databases, referencing, book a room. On the right hand side just below the quick links is information of the library’s opening hours. The body section contains a flash images and below the flash media are the university’s upcoming events and recent news. The upper side of the homepage contains a menu with navigation links for all users. This menu allows users to use the entire website and contains the following: The Find link provides users resource materials such as books, databases, and journals The Space link allows the user to search for space information in the library i.e library layout, study rooms etc The Borrow link provide the users with a page with further links for borrowing utility The Research link provide users with imprtant links with the research resources and tools that can enable the user work through the entire research process The Teaching links provide users (teachers and students) with important resources for teaching and learning The Publications link to university publications What’s on about us link is for quick reference to information for specific library user groups and general information about the library Help link aids users in need of assistance Fig *****: UTS website home page Features of the system The system’s uses icon controls which are arranged in a horizontal. The clickable icons links to a page which have categories and the categories have sub-categories which are links to further resources. The categorized contents make the system user friendly as the user does not need a lot of time to learn and familiarize himself to the system. The explanation immediately under and link give the user information in advance of what the user will get when he clicks the lick. On the right hand side of the page are the links similar to thos on the web body. This ensures quick clicks for users who are familiar with the system All the page have a consistent format. When a user clicks a link the contents are organized in tabs and expanding tabs. Arranging contents in tabs ensures that the contents does not clutter all of the browser and the user is not driven away from the page he was. The period the system takes to load is very short and thus mean the system does not consume bandwidth and can accommodate various simultenous connections. Process analysis One system process was under taken the user would be answering the questions in appendix 1. In testing the system the task set was to find a book on computer system from the database. Fig ***: Search Process diagram rewrite the query no yes Fig****: Search catalog page Fig*****: Suggested search results Usability analysis Based on the usability checklist the following finding was found. 13. Learnability The system is easy to use and the users do not require much time to get familiarize themselves to the system. The contents are categorized appropriately 14. Memorability Since the main menus are categorized based on the task classification the users could easily remember the routes taken to achieve a give task. The routes to achieving a given task are also minimum and average of 3 links. 15. Efficiency and effectiveness The system is both efficient and effective as it is able to achieve the expected goal of effectively managing the library resources 16. Error proneness The system is not prone to errors because it has been tested 17. Consistency The page layout and the web pages are consistent 18. Help System Below each link are the explanations on its functionality. A help link is available for users who might get confused on the way. Rating system On a scale of 1 to 5 rate the system, where 1 is the worst and 5 is the highest score. USABILITY ASPECT SCORE Learnability 4 Memorability 4 Efficiency 4 Effectiveness 4 Error proneness 4 Consistency 5 Help System 5 TOTAL SCORE 30 Strength Consistent layout design Use of expanding links Short steps/routes in achieving a given task Good help system Predictive search Systems comparisons Based on the usability rating system the best system is the UTS library system USABILITY ASPECT NSW Transport system Virgin airline system UTS system Learnability 3 3 4 Memorability 3 3 4 Efficiency 4 4 4 Effectiveness 4 4 4 Error proneness 4 4 4 Consistency 5 4 5 Help System 4 3 5 TOTAL SCORE 27 25 30 The UTS system emerged as the best system because of its consistency in layout, in design for instance they used large icons in the main menu which comprise both the icon and text. The icons are consistent with the user windows operating system program format which the user is familiar with. The system is also secure as the user is required to login in order to access the sensitive parts of the system. Recommendation Inspite of the UTS system emerging as the best system, the other systems also have good and modern technologies that can be incorporate in the design of the company’s website for example The electronic payment systems The clear step by step concept of guiding the user in filling a form as applied in Virgin flight system The use of tab system is a good page layout that can be considered in the website development of the company’s website The virtual assist feature is also a perfect user interaction tool that can be incorporate in the company’s website Conclusion It was very difficult to compare three different system to come up with a best system as each system is design to work best in each place of use. These systems have different design objectives. Based on the usability analysis the user can easily determine a good system based on the easy of its use. Having used the usability metric the UTS library system emerged to be easy, consistent and learnable. References Appendices Usability checklist Learnability How easy is it for a new user to understand how to act in the system Memorability How easy is it for an experienced user to remember how to act in the system Efficiency How much energy does a user need to put into the system to produce a desired result Effectiveness How well does the system support solving the problems of a user Error proneness How often does confusion within the system cause user Error Consistency How consistent is the software interface i.e layout, fonts Help System Does the website provide links to useful information when confronted with difficulties Process evaluation checklist 1. Number of routes taken to complete a task 2. Number of problems encountered in attempting each task 3. Number of participants who successfully completed a task 4. The average amount of time it takes for a user to complete each task 5. Number of times help was accessed during a task Read More