Pulls from scratch org no longer picking up declarative changes

[Dennis Wilson]

I'm very early on in using IC with SFDX and scratch orgs, so likely newbie confusion follows...

I successfully:

• spun up and connected to a scratch org and got that connected to my IC project
• associated a GitHub repo with the project
• created some custom objects and platform event objects declaratively
• pulled those scratch org changes into project
• created some Apex classes, deployed to scratch
• committed all those changes to repo

... basically everything was going smoothly -- until today.

Now when I make declarative changes to object metadata (i.e. rename, add fields, create new, ..etc.), and do a pull from the scratch org to get these changes, my project does not pick them up.

I've tried Generate Offline Symbol Table.

Whats weird (to me) is that from manually inspecting the OfflineSymbolTable.zip:  I can see the "new" object names and changes.  Its only the force-app part of the project tree that is out-of-date.

I'm missing the good 'ole Retrieve Metadata and Refresh Metadata in this new DX world...

Any tips on how to force the pull to "see" changes is appreciated.

Thanks for all you do Scott!

-Dennis

[Scott]

Hey, Dennis! So nice to hear from you. I hope you're doing well.

I think perhaps the best thing to do is isolate whether it's IC or SFDX that's causing the issue. Can you reproduce this from the CLI by running sfdx force:source:pull? If so, something is going awry with the CLI's source sync state most likely. If not, I'd ask you to enable debug logging for SFDX as documented here, reproduce the issue, and shoot me the log so I can see what's going on.

The OST is somewhat orthogonal to this in that SFDX itself has no idea that it exists. However, the fact that you see the object/field changes reflected in the OST does mean that they're available through the metadata API even if SFDX isn't "seeing" them.

Let's get that diagnostic info and then we can see what next steps will be.

Regards,

Scott

[Dennis Wilson]

OK, bottom line issue resolved.  But definitely a strange trip...

Your advice to drop down to the CLI was helpful.  That revealed an error:

>sfdx force:source:pull -f

ERROR:  Cannot retrieve translation for object:ExternalServiceOutbound.

Even with the -f (force) argument the pull would not complete successfully.

ExternalServiceOutbound was the old name of one of the objects that I had changed declaratively.

So the question then became:  why DX was stumbling on this object translation -- especially when I had not even enabled Translations.

Turns out this is a Platform Event Object (not regular Custom Object).  So something had become "wonky" in my scratch org after manipulating this Event Object, and DX was simply not happy with things.  Even a "forced" pull was not working.

With the help of my friend/co-worker (Mike Gill) we probed the SourceMember table, and found a dangling entry on that (now defunct) ExternalServiceOutbound Event Object.  Removing that problematic entry by hand, and then DX pulls started working again.

At this point I'm back in business.

So this experience has left me with these thoughts:

• Platform Events is still relatively new (and has bugs)
• DX tooling will probably uncover issues like this because of operating at the metadata level (low tolerance for org inconsistencies)
• Would have been nice if the IC plug-in had made the sfdx pull error more visible (perhaps in the IC tab?)

Thanks as always Scott.  Your commitment to your customer base is unparalleled. 

-Dennis

[Scott]

Thanks for circling back around with the solution, Dennis. Regarding the third point, was there a specific error that the force:source:pull command was emitting that IC wasn't displaying? If so, that's a bug. IC should show any error from the CLI when it's driving the CLI for embedded operation.

Regards,

Scott

[p...@proton7group.com]

Hi Dennis, I have the same issue -- Cannot retrieve translation for object:[my platform event]. Can you provide any more detail on your fix? I don't know how to probe the SourceMember table.

[p...@proton7group.com]

I tried adding **objectTranslations to my .forceignore file, but that didn't stop the issue.

[p...@proton7group.com]

BTW... this is NOT an IC issue. Definitely SFDX/CLI.

[Alex Berg]

I added a "Platform Event" in my scratch org and this "Cannot retrieve translation for object:My_Event" error has completely broken my "sfdx force:source:pull" command. :(

I hope Salesforce fixes this soon. The CLI client is not open-source, right? I found this project which is issues-only: https://github.com/forcedotcom/cli

[Scott]

Alex, if you haven't already I'd recommend that you share this on the SFDX success community. The SFDX team attends that community closely and is quite responsive to breaking issues like this.

This might also be a good time to point out some of the enhancements offered by 2.0.6.1:

http://www.illuminatedcloud.com/announcements/2061releasenotes

https://www.youtube.com/watch?v=Jt_BVrBOARM

Perhaps you can use its ability to perform a raw retrieval from you scratch org (shown toward the end of that video) to make your local project metadata whole, then see if push/pull can put things back into a good state. Maybe not, but it's worth trying real quick.

Regards,

Scott Wells

[Alex Berg]

Scott, yes it appears that that new "raw retrieve" allows me to successfully execute a retrieve operation. So that might unblock regular development.

However, I can't figure out how to include the "Platform Event" definition in the metadata it pulls down. I tried to create a placeholder for it by creating a "My_Platform_Event__e" directory and sub-directories like the "Account" object, but the raw retrieve operation always says that object doesn't exist in the target org.

I might have to migrate my code-base back to a non-DX format, but I believe Illuminated Cloud doesn't support choosing a scratch org connection for a new non-DX project. I can't create a non-DX connection to my scratch org because I don't know the password and can't reset it, and it would be nice to not recreate my metadata in a new scratch org... Anyways, just another normal day of development. :-)

[Scott]

Alex, platform events (and custom metadata types, etc.) are grouped under CustomObject metadata. You'll likely need to configure the retrieval dialog as follows:

I've selected Account there, but just find your platform event and check it to retrieve it.

Let me know if that doesn't get it for you.

You're correct that IC doesn't directly support traditional development against a scratch org, but there is a workaround. You can create another user in the scratch org with a known password, then create an OAuth connection for that user. You can then choose that OAuth connection in the same scratch org project instead of the scratch org connection and develop directly against the scratch org using source format metadata and all of the traditional IC features. Let me know if you have to go that route and what I've described doesn't make sense and/or doesn't work as advertised.

Regards,

Scott

[Alex Berg]

Oh! I didn't know that ensuring only the "Server Only" checkbox will include all the server items!

When I was troubleshooting this, I had all the checkboxes checked, but I still didn't see the Platform Event I created on the server. With just the "Server Only" checkbox checked, I can finally see the Platform Event I created. I select it and retrieve and my SFDX's "my-app/main/default/objects/My_Platform_Event__e" directory is created! Excellent!

Additionally, the SFDX "Push" button works to edit that Platform Event Metadata.

However, the SFDX "Pull" doesn't work to retrieve my metadata. So, it looks like I'll need to continue to rely on the "Raw Retrieve" functionality.

I'll look into creating an item on the SFDC success community. Thanks for the idea!

[Alex Berg]

This is the most relevant place I was able to find to post about this problem on the Salesforce community websites: https://developer.salesforce.com/forums/?id=9060G0000005iVnQAI

[Scott]

Here you go:

https://success.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F93A000000HTp1

Regards,

Scott

[Phil W-S]

Given my (couple of months) experience with SFDX, the solution to most problems (once you have the meta data locally) is to delete the scratch org and create a new one.