If a report is published once, then unticking the Publish button doesn't remove the report from the My Reports area. You must hit the Remove from Delivery button to remove it from My Reports
This is a bit unintuitive.
One issue is that if Publish is unticked on the report, there is no way to see that the report is still showing in My Reports.
I guess the use-case for it is that you want to do development on a report and not have the new version published until it is finished, but having the old results continue to display. This is complicated because the business user might not notice that the report is not being updated with new results.
Also the feature where a report will only be delivered if the object, target and source are flagged as Publish. However the report will be delivered when not flagged Publish at the higher level but is flagged at the report level, if it is executed at the report level, is overly complicated.
I think an indicator to show that a report is showing in my reports would be a good start. Another alternative would be to make Publish control visibility in My Reports, and use the already existing Report Status of Development to prevent the report from being refreshed in My Reports.
Additionally in the My Reports record, make it clear somehow that the report is not currently getting refreshed.