Uploaded image for project: 'Sakai'
  1. Sakai
  2. SAK-32769

GradebookNG spreadsheet scaling and performance issues



    • Previous Issue Keys:


      The current HTML Wicket table isn't able to scale with larger gradebooks. The page load time and scroll performance degrades as the number of students and grade items increases.

      It turns out the browser works incredibly hard to draw (and redraw) large complex tables, as things change around and over it (fixed floating columns/headers).

      For a gradebook with 480 students (only showing first page of 100) and 18 assignments with a primed cache:

      HTML: 3.6MB
      Total load time: 9.2 seconds
      Loading HTML: 2.17 s
      Scripting: 3.02 s
      Rendering: 2.52 s
      Memory usage: 330MB

      So, we’re seeing that each row is worth about 30KB. This does not bode well for larger gradebooks. If we were to remove pagination and have a gradebook with ~500 students, that’s a 20MB page to first load into the browser, initialise with JavaScript and have the browser draw over and over again.

      To replicate a spreadsheet experience, our goal is to remove pagination completely. For a gradebook with 480 students (no pagination) and 18 assignments with a primed cache:

      HTML: 16.7MB
      Total load time: 41.1 s
      Loading HTML: 10.24 s
      Scripting: 9.37 s
      Rendering: 21.13 s
      Memory usage: 500MB

      So you can imagine that scaling to 1000 users (or more likely 36 grade items), the HTML size will double, meaning it will take over a minute to get the HTML from the server before even rendering it or running our custom JavaScript over the data.

      The good news is this problem not unheard of and there are mechanisms to deal with rendering large datasets in the browser. The solution we believe best is to rewrite the existing table as a progressively rendered table. This will entail dropping the Wicket DataTable and bespoke JavaScript and replacing it with an open source JavaScript table plugin supported by a new JSON API. The plugin that best fits our requirements is https://handsontable.com/ (MIT license).

        Gliffy Diagrams



              Issue Links



                  Unassigned Unassigned
                  payten Payten Giles (Inactive)
                  0 Vote for this issue
                  6 Start watching this issue



                      Git Integration