ELabs Data Blessing Meeting Notes (16 September 2009)

Phased Implementation Plan

1. Allow blessing, everyone else looks the same
2. Transition to more aggressive blessing
3. Change data search to 'own user data + blessed data'

Notes

• Asking much less of the users than before
• Split.pl makes two data files
  • Original existing day-by-day raw data
  • Transformation of information from ST lines (ST is aggregation of various status commands)
    • Can this data can be inserted into the existing metadata file?
    • Otherwise, this generates a third file - what do we do with it?
• Regression testing needs to be done
• Turns out we can sample at specific intervals (use ST lines)

• Existing data? What can we do about it?
  • Proposal: default to seeing your own data and the semi-professional data
  • Option to see the legacy dataset

• Students could create a template without actually validating the template
  • Track over time - look at relative rates
  • Noisy data: learning methods? Shape of the curve, the height of the curve, number of pulses
  • This may give consistently bad data rather than inconsistently bad data if someone fools us

• We must assume that the geometry is accurate
  • Give them feedback in geometry? Show a picture? (JS? Dynamic?)

• Link the template to the geometry.

• New geometry -> do we need to generate a new template?

• Feedback to the user
  • Show pass/fail
  • Show P/F and explain about why there is a failure if we can
  • Review panel if the user believes that it is really good

• Data - user has to bless six files and then the automagic option becomes available

• Make a sort of PerformanceSplit.pl that does splitting and performance at the same time?

• Need a completed 6000-series manual

• Swift does not presently have a concept of cached data - so it ignores the pre-generated thresholded data

• Need to add more metadata to VDC (template-geometry mapping)