We are pleased to report that we have now concluded our investigations into the eClass outages that occurred on December 20, 21 and 22. After rigorous testing, our teams were able to pinpoint, replicate, and resolve the root cause.

As a brief summary, there was a sequence of issues that combined to cause eClass to fail:

• On December 20, in preparation for Winter term, an instructor copied the previous year’s instance of a course forward. That is standard practice, facilitates course creation tremendously, and is something we have never seen issues with in 12 years and over thousands of course sections. However, in this case, as part of the course, the instructor copied forward a broken calculation in a gradebook.
• The broken calculation contained what is known as a “circular reference”. The consequence was that the gradebook error was repeated many times as each Winter term student was added to the course. This severely bogged down the process that pulls student enrollments from PeopleSoft into eClass.
• The subsequent timeout filled a cache table in the database and resulted in a cascading failure that brought eClass down.

As a precautionary measure, the other changes and optimizations to eClass including the removal of resource intensive components such as the Level Up! block and mobile app will remain in place to maintain the integrity of the system during winter term and reviewed at a later date for future re-inclusion.

While we have resolved the root cause for this particular case, we will also be examining each of the steps in this sequence and applying corrections or monitoring to ensure that events of this nature cannot occur in future.

Thank you for your patience and best wishes for everyone in Winter term.

Regards,
IST eClass Support