Testing with new CourseSales.com versions

Tuesday, Jul 30, 2019
Home

When preparing for major software releases testing should be focused on ensuring all current functionality works as expected. When allocating time it can not be predicted when configuration changes or trouble shooting be required and how long this might take. The purpose of the testing is to reduce operational disruption. During each upgrade period the CourseSales.com technical team are available to address any issues that interrupt normal functionality, this effort is including as part of the existing license.

Broadly speaking we recommend testing is completed of the following areas of functionality:

  • Moodle interactions (login, update courses, edit courses etc)

  • Registration, each process step

  • Emails sent, PDFs created

  • List of courses, links from website to each course

  • AVETMISS reporting (interim reporting to the AVS website)

  • Any API interactions

  • CSS and Javascript appearance / functions.

The timing of each major release is dependent on stability of the system, we will wait until we are sure we have minimized the risk as much as possible. We do this by running through a number of automated and manual tests. We provide 2 weeks notice of a major release and have a stable version in the test environment for you to evaluate. Minor releases involving minimal changes can happen at any time.

We recommend a test environment of your site (separate from your website) including Moodle (if in use) that mirrors your current functionality and can hook into the test environment we provide at https://[shortname].test.coursesales.com. Much of the testing can be automated, using selenium IDE. We have scripts we can share that do this and will need to be customised. http://www.seleniumhq.org/projects/ide/

The following are details of suggested tests that we would recommend as a starting point to confirm new releases work as expected. When you sign up for a license you will be subscribed to our CourseSales.com customer release newsletter. Be sure to read up on documentation before hand. We have divided the testing functions into the following groups:

  • Administration Tests to confirm, for your course administrators, that normal add, edit and modify operations will work as expected, most of these tests will be applicable to your environment,

  • Configuration Tests - Standard evaluate standard and complex configurations which this might vary depending on your business processes and license

  • Configuration Tests - AVETMISS is for Registered Training Organisations submitting AVETMISS data to state authorities for funding or to NCVER after the year-end

  • Configuration Tests - New Customers activities that are done once when a new customer joins CourseSales.com, unlikely to be useful to customers with a license.

If you have specific price rules, complex process rules relying upon multiple responses from third parties through extranet confirmations you should probably add these to your own list. We will do these tests, and have a number of licensees who also do this testing however as CourseSales.com is often essential for your business to function we recommend you carry out these tests a) in the test environment - provided free of charge 1 week prior to a release and b) upon the release going live.

By combining testing resources of all our customers to find bugs and issues we can more easily make speedy robust multiple releases during the year, improving functionality and reliability for all. We apprecate the effort you put in to help us improve CourseSales.com.

When doing these tests you can automate the testing eg. by use Selenium - contact us for some Selenium scripts, or manually enter data. While testing look for expected appearance of content in forms and on lists on your public site, look out for expected changes when actions such as expiring courses that should result in the appearance or disappearance of registration buttons on full courses. Look for removing display of courses after marked as complete and updated discounts upon applying price rules etc.

Administration Tests

  • Create new terms: Publish > Content > terms (associate T&C to each Course Price Type),
  • Create new venues: Publish > Content > venues
  • Create new formats: Publish > Content > formats
  • Create locations: Publish > Options > locations
  • Create course descriptions: Publish > Content > Course Descriptions
  • Create PDF templates: Publish > Content > PDF template, attach to a registration process step, include within the PDF the table of catalog items ie. on invoices with cost
  • Create promotional items: Publish > Content > Promotion items
  • Add a new file: Publish > Files > New, attach to email
  • Create dropdown form with default value, then change to checkboxes or radio
  • Create process step for attendance, rule to create contact and path for course dates and documents
  • Create catalog, catalog group & link to a course master, edit form to include catalog items, view and publish
  • Confirm catalog prices come into registration form & PDF (with all description sections).
  • Create Publish > Masters > Edit > Add dates, Course Dates
  • Create Courses > Events > New, add course dates
  • Set up public pages, make enquiry and registration via public site and admin interface on each course price category
  • Visit public pages display of courses is seen, and expected document buttons exist.
  • Export course dates, documents and contacts
  • Lookup a course date, and document using unique identifiers
  • Set up kiosk login, forms and process step
  • Visit kiosk and login with correct user & mark as attended, try to login with a different user
  • Change a course date process steps, confirm change acted as expected, check against Preview mag+ button on edit page
  • Check Document Payment Summary
  • Login as trainer, and check permissions
  • Scan System > Audit for details or unusual activity and expected activity
  • Read documentation on the CourseSales.com website to confirm this reflects how functions actual work

Configuration Tests - Standard

  • Check that API calls are delivering expected values
  • Add the Course Dates to Events, manage the documents associate to the event and course dates associated to the event
  • Configure Promotional items at Location, Provider and Course Category level; confirm content displays when expected (emails/PDF/website)
  • Send message to phone via sms
  • Create Setup > Logins > New Trainer login restricted to role limitations, check these are upheld
  • Check invoices add up correctly
  • Check that specific Course Price Types on public pages display expected Terms
  • Check that specific Course Price Types on administration portal display expected Terms
  • Check extranet forms are linked from emails and that these links work, triggering expected email and attachments.
  • Check generated PDF files are attached to bookings and invoice emails
  • Check prices are updated using Price Rules, cron change and discount a course
  • Check Email and PDF forms include expected replaceable variables esp extranet forms, attached PDFs
  • Check display of forms, location of fields
  • Fields & forms: Confirm default values, content, drop downs, auto complete fields, field alignment
  • Follow the process steps for a course
  • Make an enquiry from websites, confirm HTTP_referrer is correctly recorded in administration portal
  • Do the above on Windows (IE, Firefox & Chrome) and Mac (Safari, Chrome & Firefox)
  • View public pages (lists, forms, extranet forms, kiosk) on mobile and tablets for all claimed to be supported/used.

Configuration Tests - AVETMISS

  • Add sample documents with AVETMISS data
  • Export past AVETMISS reporting period where a baseline is available - confirm NAT files from that past period matches the baseline data eg using microsoft compare function, or linux ‘diff’
  • Export all NAT files, check these are validated with NCVER

Configuration Tests - New Customers

This is ideal for those organisations who are new to CourseSales.com

Register as a student

Check the following:

  • Register on each course that varies in configuration: eg if you have 5 courses, each with a different assessment then each of these should be tested separately.
  • Course categories are accurate and complete
  • Course descriptions are accurate
  • The forms gather the correct information
  • That the mandatory fields are correct
  • Terms and conditions are correct

Assessments

When using the assessment plugin it is important to check that all assessments work as expected.

  • Assessment questions are accurate and complete
  • Expected attempts are correct
  • draft save, fail and success messages are as expected
  • If using automatic draft save this is working as expected
  • If using

Confirming outcomes

  • Confirm who will be marking outcomes: admin or trainers

  • Are trainers able to mark outcomes correctly?

  • Signup for new licensees from CourseSales.com

  • Upload a logo, upload a CSS file

  • Edit all items on a course master to use dummy data

  • Create a tax rule

  • Create Finance > Payment Method, associate to Finance > Payment Group, link to course master

  • Set up 2 export maps associate with registration, send to FTP & queue

  • Activate necessary roles for public pages, namely Web Services and Public

  • Create AVETMISS fields and forms

  • Set up Course Price Types; discount a, b & c

See Also