Tip: hiding sfdx pull (and "metadata refresh")

[Phil W-S]

I have found that sfdx pull is very problematic since:

1. It always pulls all "changes" from an org

2. These "changes" include various automatic "changes" applied by Salesforce itself, especially to profiles and permission sets, that are not relevant or valid for storing in our source tree.

I also find "metadata refresh" troublesome for similar reasons, on occasion.

As such I made sure to hide these options via File > Settings... Appearance & Behaviour > Menus and Toolbars > Main Toolbar > MainToolBarSettings > IlluminatedCloudToolbar to avoid accidental clicks on these buttons.

I always use the "Retrieve Metadata" option since I can be selective and can also use "Retrieve for Merge" to allow inspection of the metadata before merging it into our source tree.

[Dave Seligson]

The .forceignore file will tell "pull" and "push" to respectively ignore metadata items or files.

Note that, of course, if you are working with PermissionSets/Profiles you definitely want to use "Retrieve Metadata" and exclude them with the .forceignore file.

(Especially because the "pull" only pulls the changes, which would wipe out the other 98% of the file from the one that was loaded.  And if you then commit that file, guess what?  You just cleared out your repo's version, too).  "Retrieve Metadata" on a/all PermissionSet elements always pulls the full file, so git stays in sync like normal source control should work.

Just my 2c, your mileage may vary, not applicable in all states,please talk to your doctor before embarking on a new exercise plan, yada yada yada.

[Phil W-S]

Thanks for the addition. Yes, I didn't mention .forceignore. My additional two cents on that subject:

For me, the problems with .forceignore are:

1. It is somewhat poorly documented (though has been improved recently). Knowing exactly what to put in the file can be challenging; the syntax for identifying specific metadata components is simply the file path for push/deploy, whilst is just the API name and metadata type name for pull/retrieve. The documentation does specifically cover a profile example, but identifying the correct metadata type name doesn't always appear to match the folder structure or *-meta.xml file extension. Please correct me if I'm wrong.

2. As the documentation now says (it didn't used to), this is applied not just to force:source:push/pull/status but also force:source:deploy/retrieve.

What's the "so what" for point 2?

Whilst IC2 appears to continue to use the Tooling/Metadata API to deploy metadata, we have scripting that uses sfdx to specifically push and deploy our metadata in stages to newly created scratch orgs since we have a need to deploy certain metadata just the once during the creation of a scratch org, for development purposes (long story short we need to explicitly enable Shield on certain key fields on certain objects to validate that our AppExchange package is Shield compatible and that means jumping hoops when first deploying our package's metadata to a scratch org). Thus we are unable to use .forceignore to exclude our packaged permission sets and other materials since it would prevent the metadata from being sent to the scratch org in the first place.

Anything else?

The force:source:pull command is rather blunt force (pardon the pun). You cannot be selective; you'll get all metadata that has changed or been added (excluding those listed in .forceignore, of course). It's common to play on scratch orgs during development and many such changes are destined for the trash, yet force:source:pull can't be told to ignore these without manual changes to .forceignore (which you probably don't want to touch because that's often version controlled too, plus knowing just what to use for the name can be challenging as I already said).

This can easily lead to accidental and inappropriate updates of metadata in your local source tree and, without due diligence, may lead to accidental inclusion in commits to git (or other VCS). This then means wasted time reversing out such changes, which can be non-trivial depending just how late the inclusion is spotted.

More still?!

Related to Dave's comment about retrieving only changes for profiles/perm sets, I've found that retrieving profiles (and, I assume, permission sets), will only fetch data that relates to objects that you retrieve at the same time. This is where the Retrieve for Merge is really helpful because you can select all objects on the org (and/or selectively exclude or include the ones you want) and get the profile/perm set fully populated, whilst having no impact on your local metadata for these objects.

[Scott]

Another thing you can do is have IC2 not treat a scratch org as a scratch org at all by enabling Use deploy/retrieve/delete instead of push/pull on that scratch org connection. That will make the scratch org behave exactly like a non-scratch org for purposes of metadata management, using IC2's own native deployment, retrieval, and removal features. You can find more info on that here. Of course, the scratch org will still expire at some point, but then you just replace it with one with the same alias and repopulate it for the next 30 days.

Phil, I'm curious about the issues you've seen with the Refresh Metadata command. If you have a reproducible test case, please either grab a debug log for an execution of it along with expected/actual behavior or just let me know those steps such that I can reproduce them locally.

Regards,

Scott