The Effectiveness of  Online Learning

In May 2009, the U.S. Department of Education published a report on the findings of an extensive meta-analysis. (See the citation and links at the bottom.) The researchers screened over 500 articles in order to answer four research questions:

1. How does the effectiveness of online learning compare with that of face-to-face instruction?
2. Does supplementing face-to-face instruction with online instruction enhance learning?
3. What practices are associated with more effective online learning?
4. What conditions influence the effectiveness of online learning?

The researchers found:

1. Students who took all or part of their class online performed better, on average, than those taking the same course through traditional face-to-face instruction. The online and face-to-face conditions generally differed on multiple dimensions, including the amount of time that learners spent on task. The advantages observerd for online learning conditions therefore may be the product of aspects of those treatment conditions other than the instructional medium per se.
2. Instruction combining online and face-to-face elements had a larger advantage relative to purely face-to-face instruction than did purely online instruction. Because of experimental control limitations, the observed advantage for online learning in general, and blended learning conditions in particular, is not necessarily rooted in the media used per se and may reflect differences in content, pedagogy, and learning time.
3. Spending more time on task offered a greater benefit for online learners than for face-to-face learners.
4. Only two of the thirteen online implemenations surveyed had a significant effect on learning outcomes. These were use of a blended rather than a purely online approach and the expansion of time on task for online learners.
5. The effectiveness of online learning approaches appears quite broad across different content and learner types. Neither subject of instruction nor learner type (K-12, Undergraduate, and Graduate) had a significant effect on learner outcomes.
6. Whether the instructor was the same under online and face-to-face conditions did not significantyly affect the outcomes.

From a narrative review of experimental and quasi-experimental studies contrasting different online learning practices found that the majority of available studies suggest the following:

1. When a study contrasts blended and purely online conditions, student learning is usually comparable across the two conditions.
2. Elements such as video or online quizzes do not appear to influence the amount that students learn in online classes. The practice of assigning online quizzes does not appear to be more effective than other tactics such as assigning homework.
3. Online learners benefit from activities that trigger learner action or reflection and self-monitoring of understanding  (metacognition).
4. When groups of students are learning together online, support mechanisms such as guiding questions generally influence the way students interact, but not the amount they learn.

The literature review was limited to:

• studies of Web-based instruction
• studies with controlled, quasi-experimental design
• studies which examined effects only for objective measures of student learning

 The following table, taken from the study, shows examples of the interaction of three components describing online learning activities:

• Objective: Face-to-Face Alternative (Fully Online) vs. Face-to-Enhancement (Blended) 
• Type of Learning: Expository (e.g. lecture); Active (learner is active); Interactive (students collaborate)
•  Synchronous (in real time) vs. Asynchronous (time lag between communication and response)

In establishing a conceptual framework for the study, the authors identified three components of online learning and provided the following table to demonstrate their interaction.

Conceptual Framework for Online Learning

Exhibit reads: Online learning applications can be characterized in terms of (a) the kind of learning experience they provide, (b) whether computer-mediated instruction is primarily synchronous or asynchronous and (c) whether they are intended as an alternative or a supplement to face-to-face instruction.

 Citation

U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies, Washington, D.C., 2009.

Links

http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

http://learn.champlaincollege.qc.ca/PD/finalreport%20US%20dept%20of%20education.pdf

Elliot Soloway: Researcher in the Integration of Technology into Education

Your comments are welcome.

Home

adfdsaf

At the heart of the World Wide Web’s architecture is hyperlinking, a feature which gives the Web its power. Hyperlinks allow users to choose their own path around the Web. The massive database of information provided by the Web is a function of the content placed there and the web of access routes created by users as they navigate to points of interest.

Web 2.0 has discovered the user intelligence embedded in this web of access. It has answered questions such as, “What is the best site on the web on how to train a dog”?

Google’s PageRank strategy answers the question by counting up popularity votes in the form of links to a site that purports to be about dog training. The thinking is that, if other sites about dog training contain links to site X, then site X is probably a highly relevant site on the subject. Google’s relevance rating of web sites is based on the collective intelligence of web site developers.

Delicious.com is a social bookmarking site. Here you can enter links to your favorite web sites, just as you might add them to the Favorites menu in your browser. An obvious benefit is that your favorite links are available to you no matter where you are and no matter what computer you are using. But the additional benefit is collective intelligence. It is often referred to as a folksonomy, an index of useful information compiled by users rather than traditional experts such as librarians.

Amazon.com is a very successful online marketer that makes extensive use of its customers’ book choices. Each regular visitor to the site is greeted with book recommedations based on the choices of other shoppers with similar tastes. In addition, users write reviews and share their personal lists with other shoppers.

The success of Wikipedia is well known. It is an online encyclopedia where the entries can be added by any web user, and edited by any other.

Cloudmark is a collaborative spam filtering product which aggregates the decisions of email users about what is and is not spam. It can outperform systems that rely on analysis of the messages themselves.

Successful Web 2.0 sites are usually characterized by the effective use of the collective intelligence of the user base.

Your comments are welcome.

Home

In the early days of the Web (1990) interaction between browsers and servers was largely uni-directional. Information was donwloaded from the server to the local PC. Web 2.0 arrived with the shift to bi-directional communication on the Web. Many web sites interact with their users enabling them to provide valuable services, as in the case of Google and Amazon, for example.

Web 2.0 applications are not limited to the PC platform. Even the simplest web application involves at least two computers: the one hosting the web server and the one hosting the browser. And so any web application can be seen as software above the level of a single device. This could be contrasted with the more conventional single-device applications like MS Word which are installed on each local computer and must be available in different versions for PC’s and Macs.

The software used to write this blog demonstrates the nature of a web application and its machine independance. The writing and editing is controlled by software on the server and functions in the same way for all users no matter what local platform they are using. The http protocol, and the HTML language that is used to define web pages, as well as the standardized programming languages of the web (PHP in this case), give all applications the ability to function across the diverse collection of personal computers that connect to the Web.

The future may hold some significant surprises as the marriage of Web applications and more sophisticated hand-held devices gives us applications that permit effective bi-directional communication. We have already seen real time traffic monitoring, flash mobs, and citizen journalism as early examples.

Your comments are welcome.

Home

Mike Wesch at the Library of Congress

Brolsma

Original Numa Numa video

A re-mixable data source is, by and large, a web site that permits the user to combine the site’s data with information from other sources, in order to create some new informational view. The result is often referred to as a mashup.

The site developers usually provide a set of tools to facilitate the merging of external data into the information offered by their site. These tools are usually referred to under the arcane acronym, API. An Application Programming Interface is like a set of standardized connections into the web site so that mashup creators can incorporate the external material.

These video tutorials demonstrate how the Yahoo Pipes API works.

One of the most common examples of a mashup is the integration of various sorts of data, for example, real estate information, with a map. Google Maps is a good example. Users can create their own maps by adding images, YouTube videos, and other markers.

European campers have used Google maps to show the location of campsites. Then there is the Diaroogle Team whose web site offers a detailed map showing the locations of the public toilets in Manhattan.

Flickrvision is a global slideshow where photographs, as they are uploaded to Flickr, pop up on a map of the world, (in 3D if you wish) indicating the location of the photographer.

Twittervision is a variation on the Flickrvision idea.

One way to better understand the mechanics of creating a mashup is to go to Microsoft’s Popfly. Here is an example of one user’s mashing of Google maps and upcoming concert information.

The mashup is a relatively new idea and its development relies heavily on web site developers creating and enhancing the API’s that enable to combine data from different sources.

Your comments are welcome.

Home

To understand what’s meant by cost-effective scalability let’s look at some new servcices offered by Amazon.com.

It needs to be understood, first of all, that Amazon is a very successful international online marketer that has moved beyond selling books to selling almost anything, including power tools and non-perishable groceries. The result of this phenomenal growth is that Amazon has a very sophisticated IT installation that can be expanded relatively easily to enable the offering of new services.

Amazon has recently offered several new services aimed at businesses. These services fall under the heading of “cloud computing”.

In the past, and in the present for most organizations, important computer applications are installed and run in-house. What Amazon, and others, are proposing is to take these applications (assuming they are Web-based systems) and install them in the “cloud”. This means installing them in a place on the Web provided by someone like Amazon. Instead of running important applications on an in-house web server which must be maintained, programmed, and backed up, the business can install them in the virtual world of Amzon’s server farm. In this environment it is Amazon that bears the cost of maintaining an extensive hardware and software array with all of the requisite support. In addition, a client can scale his applications up or down in a matter of minutes eliminating the long delays normally associated with inhouse IT upgrades and the need to oversize to handle periodic spikes in demand. The client controls his applications remotely, through a web interface.

Amazon (and others) can now provide access to software, computing, storage, and applications from a location outside the walls of a customer’s physical site. Although it can be practical and cost effective, many organizations are still not ready to entrust storage and processing of very large amounts of data to an outside service. Despite a willingness to rely on external services for electricity, water, and Internet access, and a well established trust in banks as offsite storage facilities, the fear of exposure or loss of important data remains. The fact that Amazon is a credible vendor with a strong track record and good future prospects is beginning to address this fear.

Amzon’s “cloud” can provide a client organization with hundreds of terabytes of network storage which can accommodate even large relational databases. The cost is moderate and the business pays only for what is used. Making some assumptions about an enterprise with 5,000 employees, using Amazon cloud applications around the clock, the bill could be as low as $4,600 per month. Potentially, an organization could eliminate its large and expensive data center thereby substantially reducting its IT investment and operating costs.

These services offered by Amazon illustrate the cost-effective scalability that comes from the operation of a large, networked server farm. Companies like Amazon and Google have already created, enhanced, and support a large-scale data processing operation. The incremental costs of providing processing services to a new customer are small compared to those the customer would face in installing or upgrading an inhouse data center.

The major obstacle to this trend in IT service outsourcing lies in the fears of IT managers about data and processing security. The attactive commodity pricing of cloud computing, on the other hand, will likely go a long way to encourage IT departments to make the switch.

Your comments are welcome.

Home

Architecture of Participation is an O’Reilly concept that describes systems, such as the World Wide Web, that are designed for user contribution. The Windows-PC platform, on the other hand,  requires that application contributors have a substantial amount of programming expertise in order to navigate the many layers of software that make up the Windows operating system. The system’s architecture discourages widespread participation by people who are not experienced programmers.

The World Wide Web, on the other hand, is an example of a system designed around communications protocols. This makes it conducive to large-scale collaboration. It opens participation not just to software developers but to all users of the system. Anyone can add worthwhile components.

The Web architecture itself demonstrates the open source mentality which is the dominant mode of software development in the Web 2.0 era. HTML, the language in which web pages are written, is readily accessible to ordinary users. The “View Source” menu item found in web browsers like Internet Explorer allows full disclosure of the web page content. This is important to amateurs who can use this mechanism to learn from the work of others.

In addition, the World Wide Web was built around the concept of hyperlinking . This refers to our use of links on web pages to take us to other pages or web sites. (It is this jumping from place to place on the Internet that potentially defines a “web”-like path.) Through the use of links, web architects and web surfers define unique ways of mapping or configuring knowledge. Through user contribution the Web has become a massive global database since its inception in 1989. 

Wikipedia is a good example of database building by volunteers. There are many others including delicious.com which accumulates web users’ favourite links and amazon.com which can connect an individual’s book choices to those of other customers with similar tastes.

According to O’Reilly, “The architecture of Linux, the Internet, and the World Wide Web are such that users pursuing their own ’selfish’ interests build collective value as an automatic byproduct. [...] These projects can be seen to have a natural architecture of participation.”

Your comments are welcome.

Home

In a previous post, I discussed the evolving defintition of the word “platform”, how it now includes the entire World Wide Web. It is as though, when you sit at your computer, your platform extends off of your desk, out of the building, and across the entire Internet.

As the Web becomes the platform, it becomes the base for a plethora of new applications spawned continuously by sources all over the world. Microsoft understands this shift and knows that the days of the boxed software business are numbered. This explains its interest in buying Yahoo, a last chance to own a piece of the growing Web application market.

The distinction that is made between service and software is a bit misleading in the case of Web 2.0. The real difference lies in the source of the software and, to a real extent, in its cost. On the small platform (one ’s own computer) programs are bought and installed. This is the case for applications like Microsoft Office (Word, Excel, PowerPoint), Photoshop, video editors, etc. It is this kind of product that should experience a shrinking market in the future as a wider array of services becomes available on the Web.

The services offered tend to reflect what users really want and need. Although there is presently not an online word processing application that can match MS Word in power or sophistication there are adequate text editors in most Web applications (blogs and wikis, for example) to enable the creation of well formatted documents for the online environment.

The question of what applications computers users want to use is an interesting one. It would seem that what we have been wanting for some time are ways to network with other people, to locate others of like (or dissmilar) mind and to communicate or share information with them and perhaps to collaborate with them. These applications include flickr, delicious, myspace, facebook, YouTube, twitter and many others.

In addtion, there are many practical tools that are offered as free Web applications, such as pbwiki (free wiki site) and wordpress (free blog site). There are also applications, such as course management systems, which formerly would have been supplied by commercial vendors and now are freely provided online (edu20).

These shifts are significant for entrepreneurs. Under traditional marketing models a few potential customers might receive a free sample of a new product while most eventual users will pay for it. On the Web, this paradigm is often reversed: many users receive the product for free while a few eventual customers will pay for premium service.

Your comments are welcome.

Home