Please Share your Product Ideas with us!

All ideas are welcome. Just because the Idea doesn't make it into the product immediately does not make it a bad idea.

ETL Breakpoint

Migration Rules are very execution order sensitive. The results for a given rule are often very dependant on all the rules above having run successfully. Only in that case would the current rule be "Debuggable" because it is dependent on the state of the data at that one point in time.

Therefore it can be VERY difficult to debug a logic error in a rule unless you can STOP the execution of the rules at the precise step needing to be debugged.

Just like an IDE, we need a Breakpoint on our ETL rules. Setting a breakpoint would cause the ETL job to STOP in the middle of execution, allowing the developer to debug the RULE. Once complete, they should be able to CONTINUE the execution of the remaining tasks.

  • John Munkberg
  • Feb 23 2023
  • Shipped
  • Attach files