Workflow for multiple developers using SFDC, IC, and Github

[Michael R]

Hello!

I'm looking to get some advice on setting up our multi-user developer group.  We're using IC and github, and having difficulty conceptually connecting the pieces together.  

A few of the thoughts/confusions which we're looking to get the group's thoughts on:

• How should the initial Github repository be populated?
• Ant migration tool?
• Download "All/Package" then commit via git CLI ?
• Others?
• Illum Cloud:  
• What is the difference between a project and a module?  1-Project to 1-or-many Modules,,, but is there any advantage to ever have multiple modules for a given project?
• What "subscription" should we use once we have seeded our git repo?  
• I have seen references to avoid "All/Package" but I'm not entirely clear why that is a bad idea.
• Utilizing "package.xml" or "selected" makes sense until someone makes a change that affects metadata that has not been subscribed to.
• How to choose how little or how many things to subscribe to?
• SFDC org:
• I imagine having multiple dev orgs per developer?  
• IN this way, one org can capture work for a given feature branch
• Github:
• Branches to capture features / releases ?
• Branches to represent each environment; e.g., Test, Stage, PROD (master) ?
• Pull requests any time something merges into master.
• I've seen Altassian's write-up on using multiple repositories via fork-synching, but at the moment we're stuck with github for corp reasons.

Thanks in advance!

MIke.

[Scott]

There are quite a few questions in here, so I'll do my best to answer as many as I can in as coherent a form as I'm able, but obviously a full answer may require a few iterations of Q&A.

How should the initial Git repository be populated?

Based on your question, it sounds like you're starting with the metadata residing primarily in an existing org and you're wanting to start managing it in a version control system.  If that's the case, then yes, you'll want to retrieve the metadata that constitutes your application/customizations and check that in.  I emphasized part of that to be clear that it's very important that you carefully isolate the metadata that represents the work that, when added to a (relatively) pristine org, recreates the functionality of your application and customizations.  You don't want to pull down boiler-plate metadata such as the standard definition of Account or its default (unaltered) layouts, and you definitely don't want to pull down sample config from the org if that's present.  That's why All/Package is generally a bad idea unless you're actually using development packages.  If the org has been the primary repository of your metadata to-date, this may be an onerous exercise, but it's critically important for enabling team development in a repeatable fashion.

In Illuminated Cloud you can describe this subset of metadata in two primary ways: 1) as a package.xml which will also work with any other standard Salesforce deployment tools; 2) as a custom metadata selection via the subscription configuration screen.  I generally recommend the first option for anyone planning to do automated builds, continuous integration, and/or continuous deployment because there's a single, declarative representation of the metadata that can be used across tooling.

Once you've retrieved the correct metadata, you can just check it into version control.  In addition to the metadata itself, you can also check in most of the IDE configuration files to be shared by the team.  I would refer you to the release notes for 1.7.8.0 to understand which IntelliJ IDEA and Illuminated Cloud config files should be checked into version control and shared among team members.

I will say that for the smoothest experience sharing the IDE config through version control, choose a consistent name for each connection based on its role and not on any user-specific naming preference.  This name may be representative of the product/feature that the metadata creates, or it may be the role of the target org.  The reason for consistent naming, though, is that the connection name is stored in the config files that you'll check in, so when a new user checks out the source code and opens the IDE, he or she will be prompted to configure the named connection for his or her purposes.  For most folks that means just providing the credentials of the person's own dev org.  Once the named connection is configured, everything else will work properly and the IDE config files won't need to be modified in a user-specific fashion.

What's the difference between a project and a module?

I'll let JetBrains' own documentation on this set the stage, but I'll also talk about how these can be used specifically for Salesforce development in Illuminated Cloud.

For many people, there will be a one-to-one correspondence between projects and modules with the project really just being the necessary scaffolding around a single module, and the module being the container for the source code and the tie to the org via the selected connection.  In that scenario the project/module directory structure tends to be very simple and shallow, e.g.:

• /path/to/projectRoot/
  
• .idea/
• illuminatedCloud.xml
• ...
• IlluminatedCloud/
• connectionName/
• OfflineSymbolTable.zip
• moduleName.iml
• src/
• classes/
• pages/
• ...
• package.xml

However, there are some interesting and compelling reasons to have more than one module in a project.  The most common reason is to have convenient access to more than one "project's" worth of metadata in a single IDE window.  In that scenario multiple connections would be configured, at least one per-module, and the project/module directory structure would look more like:

• /path/to/projectRoot/
  
• .idea/
• illuminatedCloud.xml
• ...
• IlluminatedCloud/
• connectionName1/
• OfflineSymbolTable.zip
• connectionName2/
• OfflineSymbolTable.zip
• ...
• moduleName1/
• moduleName1.iml
• src/
• classes/
• pages/
• ...
• package.xml
• moduleName2/
• moduleName1.iml
• src/
• classes/
• pages/
• ...
• package.xml

Generally I'd only really recommend doing this if the modules' contents have some logical relationship with one another.  Otherwise I'd recommend having multiple projects for proper separation of concerns.

There's another use case around multiple modules in a single project that has to do with development of metadata for multiple managed packages in a single dev org, but I'm going to omit that from this description because it touches on other complexities, some of which Illuminated Cloud doesn't currently fully address.

Multiple dev orgs per-developer?

For team development, I highly recommend at least one dev org (ideally a partner dev org if you have access to them) per-developer.  That way each developer checks out the metadata from version control and deploys it into his or her own dev org, thereby avoiding contention with other developers except in version control where merging/integrating is already a first-class consideration.  For what it's worth, this is the exact model that Salesforce is promoting with their new Salesforce DX feature except that DX scratch orgs are much better suited to the task than dev orgs because of their ephemeral nature.  Until those are fully-available, though, (partner) dev orgs are your best option.

There is one notable complexity with dev orgs, though.  For a truly repeatable process you'll likely want to be able to remove all metadata from your dev orgs occasionally to ensure that you don't pick up transient changes.  For example, imagine that mid-sprint someone creates an Apex class called MyRealyCoolClass and checks it in, then all the other developers pick up the change from version control and deploy it into their respective dev orgs.  A day later the developer realizes that the name has a typo, renames it to MyReallyCoolClass, and checks that in.  Now every dev org has both MyRealyCoolClass and MyReallyCoolClass.  I can guarantee that someone is accidentally going to check the former back into version control!  Unfortunately there's no out-of-the-box "make clean" for a Salesforce org, though there are several scripts out there that facilitate this by creating a destructiveChanges.xml based on what's in your org and deploying it, thereby effectively razing your org to the ground.  It's probably a good idea to do this occasionally to safeguard against cruft accrual.  Salesforce DX scratch orgs avoid this by being ephemeral...no need to wipe an org clean...just discard it and create a new one!

Because some development occurs in the IDE--Apex, Visualforce, Lightning, etc.--and some occurs in the org--custom object/field modeling, layout management, etc.--developers will not only be deploying local changes into the org, but they'll also need to retrieve changes made in the dev org into the local filesystem before checking into version control.  Illuminated Cloud's Retrieve and Merge feature allows the org metadata to be carefully reconciled with the local metadata to make sure that things are as-desired before committing them.

In this model, version control should be considered the single source of truth for the metadata.  Developers check out the latest valid copy from version control, deploy into their own dev orgs to work, retrieve from those orgs to get the complete set of metadata, and when ready, commit those changes to version control to share them with the team and to create the end product.

Version control and branching strategies?

There are probably as many branching strategies as there are developers so I won't spend much time here advocating one or the other.  Many good documents have been written on successful branching and workflow strategies with VCS in general and with Git in particular.  The good news is that once you align on version control as the single source of truth for the metadata, you can really apply any of those strategies to Salesforce development that you want.

Okay, let's start with that.  After your team digests that info, please let me know if you have additional questions or concerns.

Best regards,

Scott

[Christian Anderson]

Hi Scott, this post is extremely helpful.  I'm getting ready to check-in the .iml file and the illuminatedCloud.xml file into version control.

1) In Preferences -> Languages & Frameworks -> Illuminated Cloud I have 4 connections setup named: devsandbox-christian, devsandbox-luke, fullsandbox-uat, production

However, in illuminatedCloud.xml I only see one connection listed.  Should I see all my configured connections in the .xml so they are shared through version control?  Obviously the user/pass would not be shared.

    <option name="connections">
      <list>
        <IlluminatedCloudConnection>
          <option name="connectionName" value="devsandbox-christian" />
          <option name="defaultModuleName" value="Aquarius-Capro" />
          <option name="offlineSymbolTablePath" value="$PROJECT_DIR$/IlluminatedCloud/devsandbox-christian/OfflineSymbolTable.zip" />
        </IlluminatedCloudConnection>
      </list>
    </option>

 2) In the .iml file, I notice that the connectionName is for my sandbox, devsandbox-christian.

    <facet type="IlluminatedCloud" name="Illuminated Cloud™">
      <configuration>
        <option name="connectionName" value="devsandbox-christian" />
        <option name="moduleContents">
          <ModuleContents>
            <option name="contentSelectionType" value="PACKAGE_XML" />
            <option name="manifest">
              <Manifest />
            </option>
            <option name="packageXmlRelativePath" value="src/package.xml" />
          </ModuleContents>
        </option>
      </configuration>
    </facet>

When I check it in, other developers will want this to be for their sandbox. You mention, "choose a consistent name for each connection based on its role and not on any user-specific naming preference."  Would you suggest that I have a generic connection named "MySandbox" and then each developer update the credentials for their sandbox?

I'm trying to figure out how to share all the connections to UAT, PROD and each dev sandbox through version control.  And at the same time each developer will want their project connection pointed to their personal dev sandbox.  Maybe I'm a little confused and that's not possible.

[Scott]

Christian, in 1.7.8.0 I changed things so that your connection configuration is stored at the application level in ~/.IntelliJIdea<version>/config/options/illuminatedCloud.xml.  That file and your per-project .idea/workspace.xml file contain user-specific information and are not intended to be checked into VCS.  You should check in your .idea/illuminatedCloud.xml and all *.iml files, and you can optionally check in your OfflineSymbolTable.zip if you want.

The one connection config that you're seeing in your .idea/illuminatedCloud.xml is actually an association between the project (and a specific module) and one of the application-level connections.  That's why you don't see things like usernames or passwords in that file, but if you look in the application-level config file you'll see that info.  That's what makes it safe to check .idea/illuminatedCloud.xml into version control.

As for connection names, I would recommend that you name them based on concepts shared across all developers on the team.  Stage names, product/feature names, and namespaces would be common conventions.  Then assuming folks on your team have the same general workflow, each team member would just need to provide the credentials for his or her own org that plays the role of that stage/product/feature/namespace during development.

Unfortunately finding a way to segregate user-specific and user-independent config proved challenging.  I wanted to have a balance so that the shared config was genuinely valid for being shared (i.e., no user-specific secrets or settings) while still making it complete enough so that on-boarding new team members with the files that can be checked into version control would be relatively simple.

I totally understand if this isn't clear.  Let me know if not and we can work through a couple of concrete examples.

Regards,

Scott

[Christian Anderson]

Thanks Scott.  That was helpful.  I think I've got it.