Learning Management System (LMS) Evaluation - Week #6
LMS Evaluation - Week #6 Introduction
Welcome to the Week #6 Weebly Web Page review of Learning Management Systems (LMS): LMS #1 - Brightspace and LMS #2 – Canvas. Learning Management Systems are software applications for the administration, documentation, reporting, tracking, and delivery of training programs and educational courses.
This week’s evaluation assumes a different approach. Instead of providing a first-hand evaluation of specific LMS features and functionality, this week’s review analyzes more subjective factors of each system.
Welcome to the Week #6 Weebly Web Page review of Learning Management Systems (LMS): LMS #1 - Brightspace and LMS #2 – Canvas. Learning Management Systems are software applications for the administration, documentation, reporting, tracking, and delivery of training programs and educational courses.
This week’s evaluation assumes a different approach. Instead of providing a first-hand evaluation of specific LMS features and functionality, this week’s review analyzes more subjective factors of each system.
The majority of this evaluation is based in part on interview feedback from university IT administrators with experience implementing new learning management systems into their school’s online architecture. Internet research of each subject area is also used for supporting analysis, but not in a manner that would drive decisions since internet research can often be biased or lack research rigor. Both first-hand account and internet research are important considerations when providing a well-rounded view of LMS features beyond software functionality for final recommendation to a university executive council.
Software is only as good as the company that supports it. Factors involved with this analysis include corporate strength and viability, customer analysis, feedback, and technical support, training support to the customer, analytical data provided by the software, and overall package costs. These factors help provide a big-picture view beyond features and functionality.
Software is only as good as the company that supports it. Factors involved with this analysis include corporate strength and viability, customer analysis, feedback, and technical support, training support to the customer, analytical data provided by the software, and overall package costs. These factors help provide a big-picture view beyond features and functionality.
Week #6 - Behind the Scenes LMS Evaluation
Week 6 reviews supporting elements institutions should consider when performing product analysis of a new learning management system. These elements include:
1. Customer Satisfaction Level
2. LMS Training
3. Technical Support
4. Costs
5. Analytical Data Provided from LMS
Week 6 reviews supporting elements institutions should consider when performing product analysis of a new learning management system. These elements include:
1. Customer Satisfaction Level
2. LMS Training
3. Technical Support
4. Costs
5. Analytical Data Provided from LMS
1. Customer Satisfaction Level
Customer perceptions of Brightspace are generally high as noted from first-hand administrator feedback. Brightspace has been in the market for several years, and has used that time to hone its system to meet customer needs. Positive feedback involves available features and capabilities. One IT administrator with significant Brightspace experience praised its customer support, reliability, and third-party tools. Negatives included the Brightspace online community which contains a significant amount of information, but is poorly organized.
A 2016 survey of satisfaction level by the website Eduventures illustrates Brightspace (D2L) users are 10% extremely satisfied, 40% satisfied, and 50% indifferent compared to other LMS providers.
Customer perceptions of Brightspace are generally high as noted from first-hand administrator feedback. Brightspace has been in the market for several years, and has used that time to hone its system to meet customer needs. Positive feedback involves available features and capabilities. One IT administrator with significant Brightspace experience praised its customer support, reliability, and third-party tools. Negatives included the Brightspace online community which contains a significant amount of information, but is poorly organized.
A 2016 survey of satisfaction level by the website Eduventures illustrates Brightspace (D2L) users are 10% extremely satisfied, 40% satisfied, and 50% indifferent compared to other LMS providers.
2. LMS Training
LMS training involves both training for students and faculty. Training, like any other feature, can be added to a LMS contract at various levels of quantity and support. One IT administrator reported that Brightspace training and tutorials was exceptionally well done for students, but not so much for administrators. This administrator also reported that universities typically develop their own training at various levels to train staff based on beta trial use. Brightspace offers a community website to assist with training, but it noted under Customer Satisfaction that the community is not well organized.
Brightspace does offer third party integration such as Atomic Learning, which offers various user and instructor training tutorials, videos, and step-by-step web instructional web pages.
LMS training involves both training for students and faculty. Training, like any other feature, can be added to a LMS contract at various levels of quantity and support. One IT administrator reported that Brightspace training and tutorials was exceptionally well done for students, but not so much for administrators. This administrator also reported that universities typically develop their own training at various levels to train staff based on beta trial use. Brightspace offers a community website to assist with training, but it noted under Customer Satisfaction that the community is not well organized.
Brightspace does offer third party integration such as Atomic Learning, which offers various user and instructor training tutorials, videos, and step-by-step web instructional web pages.
3. Technical Support
Technical support with any LMS is a feature offered in various levels. With Brightspace, technical support comes in various forms of LMS migration, training, issues/problems via help desk 24-hour services. One IT administrator using Brightspace indicated their university paid an up-charge for this service to receive better service and responsiveness. This administrator indicated that Brightspace support was one of the more positive features of the LMS.
A web search for Brightspace technical support yields numerous hits offered by the company itself. User reviews of technical support from the website Software Advice indicated positive reviews. One reviewer stated “The support team is very fast in replying to any issues that may arise”. Another review offered an opposing view on implantation: “Reporting and integration needs improvement. Implantation takes longer than other LMS”. Support for downtime varies since Brightspace is often hosted locally on a university’s server, and dependent on the university IT staff’s ability solve issues.
Technical support with any LMS is a feature offered in various levels. With Brightspace, technical support comes in various forms of LMS migration, training, issues/problems via help desk 24-hour services. One IT administrator using Brightspace indicated their university paid an up-charge for this service to receive better service and responsiveness. This administrator indicated that Brightspace support was one of the more positive features of the LMS.
A web search for Brightspace technical support yields numerous hits offered by the company itself. User reviews of technical support from the website Software Advice indicated positive reviews. One reviewer stated “The support team is very fast in replying to any issues that may arise”. Another review offered an opposing view on implantation: “Reporting and integration needs improvement. Implantation takes longer than other LMS”. Support for downtime varies since Brightspace is often hosted locally on a university’s server, and dependent on the university IT staff’s ability solve issues.
4. Costs
With any LMS, you pay for levels of service that include training, integration, and support costs. According to an IT administrator whose university uses Brightspace, the overall costs depends on features and feature sets the institution is willing to pay for combined with length of contract. The longer the contract, the lower the cost. Costs also vary depending on how the LMS is hosted. If hosted locally on a university server, in-house IT support must also be factored in the overall cost, which could be higher than a cloud only hosted service.
Another university IT administrator who compared Brightspace to other LMS services while they were performing LMS evaluation and found that Brightspace was priced higher than other LMS companies in apples to apples comparisons. Additionally, Brightspace plans on moving exclusively to a cloud based service in the future. Universities who currently host locally will eventually be required to move to the Brightspace cloud service at a higher cost passed on to the customer.
Brightspace offers it's service across various platforms including tablet and phone at no charge, but is not as user friendly as other learning management services.
With any LMS, you pay for levels of service that include training, integration, and support costs. According to an IT administrator whose university uses Brightspace, the overall costs depends on features and feature sets the institution is willing to pay for combined with length of contract. The longer the contract, the lower the cost. Costs also vary depending on how the LMS is hosted. If hosted locally on a university server, in-house IT support must also be factored in the overall cost, which could be higher than a cloud only hosted service.
Another university IT administrator who compared Brightspace to other LMS services while they were performing LMS evaluation and found that Brightspace was priced higher than other LMS companies in apples to apples comparisons. Additionally, Brightspace plans on moving exclusively to a cloud based service in the future. Universities who currently host locally will eventually be required to move to the Brightspace cloud service at a higher cost passed on to the customer.
Brightspace offers it's service across various platforms including tablet and phone at no charge, but is not as user friendly as other learning management services.
5. Analytical Data Provided from LMS
Brightspace provides a robust suite of analytics, known as Brightspace Insights. Local university professors and IT administrators admit that even though the analytics are available, they cannot be used across various colleges within the university to gain a strategic sight picture.
The same IT administrator stated that detailed analytics are missing in Brightspace, and are not structured in a hierarchical fashion. Big picture analytics are stated to be “flat”, and only available in the form of terms and courses. In conclusion, even though analytics are available in Brightspace, it’s an institutions choice regarding the extent of use.
Brightspace provides a robust suite of analytics, known as Brightspace Insights. Local university professors and IT administrators admit that even though the analytics are available, they cannot be used across various colleges within the university to gain a strategic sight picture.
The same IT administrator stated that detailed analytics are missing in Brightspace, and are not structured in a hierarchical fashion. Big picture analytics are stated to be “flat”, and only available in the form of terms and courses. In conclusion, even though analytics are available in Brightspace, it’s an institutions choice regarding the extent of use.
1. Customer Satisfaction Level
Canvas by Instructure is a fast mover in the LMS market, with an intuitive interface as one of its best attributes. Canvas favors well based on internet reviews, receiving praise for its user interface use of tabs and contextual menus, platforms for sharing class modules, integration, and course assembly tools.
IT administrators give Canvas high marks for its system up time, which is cloud based hosted by Amazon web services. One administrator reported that Canvas offers 99.95% up time, which is embedded in its service agreement. This administrator reported a reliability rating of less than 6 hours of system down time over 5 outages totaling 6 hours in three years. Canvas also solves technical issues by fielding version updates every two to three weeks.
Canvas rated high in customer satisfaction with this administrator and reported similar results based on student and faculty feedback. The image on the right illustrates Canvas’ rise in the market share as of Spring, 2017 to become #2 in LMS overtaking Moodle in terms of institution and student enrollment.
Canvas by Instructure is a fast mover in the LMS market, with an intuitive interface as one of its best attributes. Canvas favors well based on internet reviews, receiving praise for its user interface use of tabs and contextual menus, platforms for sharing class modules, integration, and course assembly tools.
IT administrators give Canvas high marks for its system up time, which is cloud based hosted by Amazon web services. One administrator reported that Canvas offers 99.95% up time, which is embedded in its service agreement. This administrator reported a reliability rating of less than 6 hours of system down time over 5 outages totaling 6 hours in three years. Canvas also solves technical issues by fielding version updates every two to three weeks.
Canvas rated high in customer satisfaction with this administrator and reported similar results based on student and faculty feedback. The image on the right illustrates Canvas’ rise in the market share as of Spring, 2017 to become #2 in LMS overtaking Moodle in terms of institution and student enrollment.
2. LMS Training
Canvas offers similar training options to Brightspace, using both proprietary built in training and third-party applications such as Atomic Learning. For feature-specific training, one IT administrator indicated that their university in-house training, also similar to what users of Brightspace offer. This administrator did indicate that since Canvas frequently fields new features every 2-3 weeks, Instructure (Canvas) offers webinars ahead of new feature implementation.
A significant selling point in regard to training is the intuitive nature of Canvas. Canvas promotes accomplishing tasks with less “clicks” than competitor LMS, which reduces training time and increases student/faculty integration and use.
Canvas offers similar training options to Brightspace, using both proprietary built in training and third-party applications such as Atomic Learning. For feature-specific training, one IT administrator indicated that their university in-house training, also similar to what users of Brightspace offer. This administrator did indicate that since Canvas frequently fields new features every 2-3 weeks, Instructure (Canvas) offers webinars ahead of new feature implementation.
A significant selling point in regard to training is the intuitive nature of Canvas. Canvas promotes accomplishing tasks with less “clicks” than competitor LMS, which reduces training time and increases student/faculty integration and use.
3. Technical Support
Canvas technical support is reported as one of the best features according to one university IT administrator. Reasons include an up-time rate of 99.95% guaranteed, with a service credit paid back to the university if not met. Like Brightspace, Canvas offers a help desk feature available to both faculty and students, but Canvas offers subsequent notification to university administrators for follow-up help to ensure problems are resolved.
Another advantage Canvas has over other LMS is technical support is an all-inclusive feature in the total costs of the system, which includes storage, servers, access control, and staff support due to its cloud-based platform.
Canvas technical support is reported as one of the best features according to one university IT administrator. Reasons include an up-time rate of 99.95% guaranteed, with a service credit paid back to the university if not met. Like Brightspace, Canvas offers a help desk feature available to both faculty and students, but Canvas offers subsequent notification to university administrators for follow-up help to ensure problems are resolved.
Another advantage Canvas has over other LMS is technical support is an all-inclusive feature in the total costs of the system, which includes storage, servers, access control, and staff support due to its cloud-based platform.
4. Costs
According to a local university IT administrator whose university implemented Canvas, the key to cost analysis is comparing on-premise to cloud based systems such as Canvas. When making the comparison, all associated services and features must be factored in. These features include storage, servers, access control, and staff support. A cost reduction is also realized when a university is no longer managing the system locally.
The same administrator also stated their university reduced cost by moving their LMS to Canvas in terms of manpower, local storage, server, and software. He summarized that costs for on-premise vs. costs for Canvas in cloud saved his university $75K annually. However, he advised that the decision to move to Canvas was not made on the basis of money or technology alone, but ultimately by student and faculty feedback.
Canvas excels at offering services across various device platforms that work seamlessly with mobile devices such as phone and tablets at no additional charge to the customer.
According to a local university IT administrator whose university implemented Canvas, the key to cost analysis is comparing on-premise to cloud based systems such as Canvas. When making the comparison, all associated services and features must be factored in. These features include storage, servers, access control, and staff support. A cost reduction is also realized when a university is no longer managing the system locally.
The same administrator also stated their university reduced cost by moving their LMS to Canvas in terms of manpower, local storage, server, and software. He summarized that costs for on-premise vs. costs for Canvas in cloud saved his university $75K annually. However, he advised that the decision to move to Canvas was not made on the basis of money or technology alone, but ultimately by student and faculty feedback.
Canvas excels at offering services across various device platforms that work seamlessly with mobile devices such as phone and tablets at no additional charge to the customer.
5. Analytical Data Provided from LMS
Canvas analytics evaluate individual components of a course to help evaluate student performance and whole course evaluation. Analytics, along with e-portfolio, are included to all Canvas service users and are accessed from the dashboard view. Users can also incorporate Google analytics as a third-party platform for additional analysis.
A university IT administrator whose institution uses Canvas states analytics are useful for administrators across the software, but are not designed for advanced accreditation requirements as they lack statistical significance for multiple data sets
Canvas analytics evaluate individual components of a course to help evaluate student performance and whole course evaluation. Analytics, along with e-portfolio, are included to all Canvas service users and are accessed from the dashboard view. Users can also incorporate Google analytics as a third-party platform for additional analysis.
A university IT administrator whose institution uses Canvas states analytics are useful for administrators across the software, but are not designed for advanced accreditation requirements as they lack statistical significance for multiple data sets
LMS Weekly Recommendation
This week’s evaluation involved behind the scenes evaluation of LMS factors not gained from a hands-on approach. While some internet research was used, this method of evaluation was largely avoided due to possible product bias combined with personal opinion resulting in information of unknown credibility. The majority of analysis used this week relied on first-hand accounts from university IT professionals personally involved in the research, testing, and implementation of both Brightspace and Canvas at their respective institutions.
From these accounts, training for both students and staff along with LMS analytic features seemed to be about the same for both systems. Training is available for both, but relies heavily on levels of institutional commitment for in-house training of both staff and students. This level of commitment appears to be the same for the use of analytics. While available for both platforms, application appears limited on a strategic level.
This week’s evaluation involved behind the scenes evaluation of LMS factors not gained from a hands-on approach. While some internet research was used, this method of evaluation was largely avoided due to possible product bias combined with personal opinion resulting in information of unknown credibility. The majority of analysis used this week relied on first-hand accounts from university IT professionals personally involved in the research, testing, and implementation of both Brightspace and Canvas at their respective institutions.
From these accounts, training for both students and staff along with LMS analytic features seemed to be about the same for both systems. Training is available for both, but relies heavily on levels of institutional commitment for in-house training of both staff and students. This level of commitment appears to be the same for the use of analytics. While available for both platforms, application appears limited on a strategic level.
Canvas garnered the most positive feedback in terms of cost, implementation, technical support, but most importantly, integration and acceptance by university staff and students alike. Shining features for Canvas include it's commitment to "less clicks" which speaks to it's intuitive nature, it's cloud-based platform hosted by Amazon web services, which promises 99.95% up-time, and commitment to wireless and mobile device applications for phones and tablets.
On August 2nd, 2017, PRNewswire reported that Canvas was named the best K-12 LMS solution of 2017 as part of the annual Software and Information Industry Association (AIIA) CODiE Awards, beating out five other finalists.
Week #6 Recommendation:
On August 2nd, 2017, PRNewswire reported that Canvas was named the best K-12 LMS solution of 2017 as part of the annual Software and Information Industry Association (AIIA) CODiE Awards, beating out five other finalists.
Week #6 Recommendation: