Testing

Overall approach

Given the small and ad hoc nature of the scripts developed for this project a full, formal test suite has not been written.

Scripts were developed iteratively, making use of suitably Useful forums, with interim results and final outputs manually reviewed.

Useful forums

Using the original Measuring qualification effects project’s data:

  • M811 15K Organisation and scope (Tutor - Alan Watson) is good for testing because it is small enough for manual review of the results, and contains multiple postings on one day in varying quantities. See Sample posts data.
  • M811 15K Tutor group (Mark Slaymaker) is good for testing because it has no posts.

Sample posts data

M811 15K Organisation and scope (Tutor - Alan Watson) forum_filter_posts.csv

PostID,UserID,Created
11923604,769139,2015-11-09
11928481,753224,2015-11-10
12147618,720906,2015-11-28
12183408,775978,2015-12-02
12236914,773922,2015-12-08
12237065,773922,2015-12-08
12266691,775978,2015-12-12
12325433,775978,2015-12-21
12333636,733659,2015-12-24
13042838,775978,2016-03-14
13045356,733659,2016-03-14
13182646,775978,2016-03-31
13182099,760136,2016-03-31
13182893,775978,2016-03-31
13188891,733659,2016-04-01
13194762,760136,2016-04-01
13213578,775978,2016-04-04

Quality assurance

All scripts have been successfully evaluated against:

† flake8-import-order incorrectly wants nltk, a third party import, imported before local imports.

‡ flake8-import-order correctly wants imports in alphabetical order, which I occasionally ignore for html imports to aid readability without affecting functionality.

Also, I generally follow, but not totally, a code line length of 79 characters or less, and a comment line length of 72 characters or less. Again choosing readability and the Zen of Python in line with Python’s approach to standards over dogmatic purity.

Sample doctest

Should you with to pursue a more formal test suite, see scripts/test_demo.py for:

Note, this is a relatively simple demonstration. The tests for many functions will require extensive mocking.