Custom Grouping of Manual Rules

For complex objects it is (unfortunately) common to have a long string of rules in your ETL build. For example, at Metso Customer Master has rules that do tasks like:


  • Copy data from Semarchy files (used for dedupe)

  • Insert rows into central XREFs

  • Set load type fields (Create, Extend, Update, etc.) - needed for DMC Staging Tables

  • Generate new customer IDs based on NRIV

  • Derive KNVI records off KNVV and SAP config

  • Update fields to reflect the data that exists in production instead of legacy/mapping (if dedupe results say it's already in prod)

  • etc.


Each of these bullets may have 5 or 10 individual rules to actually go perform the action needed. As a result, you end up with a massively long ETL that can be quite hard to understand what everything is doing (even when you built it all!) and very challenging to debug. If I want to debug the 'Set load type fields' tasks above, and that task has ten manual rules, I don't really have a great way to do this short of clicking every rule one by one in order ten times.


ADMM does have the ability to group like rules together under the Action column, i.e. Source Transform explodes out to all your delete/update/insert rules, Transform Reports explodes out to refresh all reports upon ETL execution, etc. but those only work for pre-delivered sets of tasks. It would be awesome to be able to create custom groupings of like tasks. This would make complex objects much more tenable to understand, debug, and support.

  • Peter Berke
  • Aug 2 2024
  • Attach files