This might be slightly off-topic, but certainly Git is version control for many companies' DITA implementations.

Short version: What tactic have you found best to minimize merge conflicts for new technical writers working in Git?

Gruesome detail: I am working with a client who stores their DITA content in GitLab. They are using SourceTree as a client. Before I was brought on-board, there was turnover and the team reduced to a manager and two technical writers, all brand-new to DITA, Git, and version control. Infrastructure engineers set up an automated build for them and recommended that they use the following practices:

• A development branch that is merged into master every two weeks when new content is delivered to the public-facing Web site.
• "Feature" branches for technical writers to use to work on new content that might span several development cycles. These branches are created from the most recent development branch.

These feature branches are problematic. Often by the time that the technical writer wants to merge them into the current development branch, there are many merge conflicts. Or the technical writer wants to selectively add in some (but not all) changes that have been made in the feature branch.

I'm trying to think of processes this team can use that will minimize merge conflicts. Here are some of the approaches I've considered:

1. Update the feature branch frequently by pulling the current development branch into the feature branch.
   
2. Leave the feature branch alone. Do not update it by pulling the content from the the current development branch into the feature branch. When it comes time to merge the feature branch into the development branch:

1. Try to do it automatically.
2. If there are significant merge conflicts, build a "patch of modified files" by diffing 1) the feature branch as it was first created, and 2) the feature branch at its current point in time. Apply the patch to the current development branch.

Have I missed something? Any advice? I have not been able to think of any non-manual solutions to the "I want to merge some but not all work from my feature branch in the development branch. And a CCMS is not an option.

We use GIT in a team of tech writers.  We do a few things...

• Division of Labor: Different members own different parts of the doc set.  This minimizes cross-talk.  But note, it does not ELIMINATE it.
• Communication: To address cross-talk (multiple people making different changes for different reasons on the same file at the same time), we communicate.  We keep track of who is working on what sections of a book, and we ASK before blithely jumping into a file.
• Push Feature Branches: If you're going to be working on a feature branch for any amount of time, push it out so everybody else can see it.  Then, if cross-talk must arise, we can talk in the same feature branch.  We still have a sense of the owner or master of that branch, and that person will ultimately merge it into develop.
• Pull from develop daily, if not more often.  Come on...  This is standard GIT practice. 

With these practices, we usually live conflict-free.  We have seen conflicts in map files and in data files that hold conref sources.  But these are usually so discrete the conflicts are rare, and if they do arise, they are easy to resolve.

I'll point out that we do docs as code, in that we deliver our source as part of the product.  That means we not only have to merge into the product repository, but we have to cherry-pick into different branches of the product repository.  So far so good!

Hope this helps!

Hi Kris,

Could you please let us know why the technical writers are supposed to do check-ins on a developer/feature branch?
This sounds to me like a very risky situation in the first place.
I am not familiar with Git though, but would it be possible to set up a folder/repository for the writers where they can write independent from the developer/feature branches? And maybe the developer branches need to make references to this folder/repository to avoid these conflicts? I've found something about using Git submodules in such a case: https://stackoverflow.com/questions/36554810/how-to-link-folder-from-a-git-repo-to-another-repo
In Subversion, our tech writers have their own folder, but they work in same repository as the developers. So we can at least use svn:externals.

Hope this helps somehow,
Christina

Christina, even if you have a separate repository for your documentation source, GIT divides it into a main branch, a develop branch, and then practice is to use various feature branches for your personal sandboxes.  Often (as is the case in our environment) you never see or know about main.

One thing I didn't address is merging a SUBSET of your feature branch into develop...  Wow...  That never even occurred to me.  I guess I would handle that by controlling my commits to my local version of the feature branch, and only push out committed changes to the remote version of the feature branch.  And then I would merge from the remote version, not the local.  Caveat here...  Our environment is set up with GitLab to handle merges, and it requires (or maybe just REALLY encourages) that you merge from a remote branch.  So that kind of split is possible. 

But I don't recommend it, for a number of reasons:

• You're not in sync with anybody else who might be using that remote version of the feature branch
• You're not in sync with what you will merge into develop -- This means if you test your work, you're not testing what you're merging!
• It doesn't feel good to me...  I like a singular concept of "ready to go".

This might be slightly off-topic, but certainly Git is version control for many companies' DITA implementations.

Short version: What tactic have you found best to minimize merge conflicts for new technical writers working in Git?

Gruesome detail: I am working with a client who stores their DITA content in GitLab. They are using SourceTree as a client. Before I was brought on-board, there was turnover and the team reduced to a manager and two technical writers, all brand-new to DITA, Git, and version control. Infrastructure engineers set up an automated build for them and recommended that they use the following practices:

• A development branch that is merged into master every two weeks when new content is delivered to the public-facing Web site.
• "Feature" branches for technical writers to use to work on new content that might span several development cycles. These branches are created from the most recent development branch.

These feature branches are problematic. Often by the time that the technical writer wants to merge them into the current development branch, there are many merge conflicts. Or the technical writer wants to selectively add in some (but not all) changes that have been made in the feature branch.

We need a bit more information on the workflow here. If changes happen only in a directory dedicated to documentation where only writers add content, that means that the changes introduced over time are touching the same parts of certain files. There are several causes and ways to deal with that:

• The tools to edit the files may be upsetting the whole files, even when changing a small chunk of the file. Git is a version control system which is mainly line based. So, it is very important that the xml be formatted with a semantic setup matching as much as possible a line by line setup and that editing tools just change the minimum number of lines. Please also note, that indentation can be a big cause of conflicts if writers don't follow exactly the same formating rules. Diffing the changes before committing should prevent big changes to inadvertantly land into the main repo.
  
• Writers have to introduce changes to the same files for small changes. Git also has a strong opinion on files as units of content. The solution is more a matter of dispatching information in different files to minimize common overlapping changes (architecture of repo). Don't be afraid to multiply the number of files instead of cramming as much as possible in one big container file. The one big container's approach with XML files can lead to big issues because it can make Git make merge errors by tricking it with similar context lines: repeated structures of lines in the file may confuse the merge algorithm.
  
• The same way files are unit of data, branches must be unit of changes. Branches can only be merged at once. So the rule is : one branch, one feature. Note that workflows with feature branches require that files are really split into the correct granularity of information, so that different branches don't introduce changes into the same files.
  

I'm trying to think of processes this team can use that will minimize merge conflicts. Here are some of the approaches I've considered:

1. Update the feature branch frequently by pulling the current development branch into the feature branch.
   

This is recommended practice.

1. 
   
2. Leave the feature branch alone. Do not update it by pulling the content from the the current development branch into the feature branch. When it comes time to merge the feature branch into the development branch:

1. Try to do it automatically.
2. If there are significant merge conflicts, build a "patch of modified files" by diffing 1) the feature branch as it was first created, and 2) the feature branch at its current point in time. Apply the patch to the current development branch.

This can be done by pulling the development branch into the feature branch first, solve conflicts then merge back feature branch into develop branch.

Have I missed something? Any advice? I have not been able to think of any non-manual solutions to the "I want to merge some but not all work from my feature branch in the development branch. And a CCMS is not an option.

--
Best,
Kris

Kristen James Eberlein
Chair, OASIS DITA Technical Committee
Principal consultant, Eberlein Consulting
www.eberleinconsulting.com
+1 919 622-1501; kriseberlein (skype)

Thank you, Chris, Alastair, and Jean-Noël for your feedback.

I do want to set some additional context here:

• I am familiar with and comfortable with Git, although it is new to the technical writers on this project. (And perhaps frightening to the manager.) I started using Git when the DITA-OT project moved to it from SVN, and I'm still grateful for all the help that Jarno Elovirta, Robert Anderson, and Roger Sheen gave me as I learned Git.
  
• Yes, good practices for Git are just standard, good software development practices: Keep feature branches discrete; update feature branches by pulling from the current development repository frequently; commit thoughtfully and frequently with good commit messages, so that cherry picking is a possibility.
• For this project, the technical writers are using oXygen, and we can control formatting through common project files to reduce conflicts.

That said, version control with Git can have a steep ramp-up for technical writers. Any thoughts about how to best minimize that? Useful resources?

And how to best plan a process for the scenario in which a writer wants to merge some (but not all content) from a feature branch into the development branch? In the absence of granular commits with descriptive commit messages, cherry picking is a not a realistic option.

So far I've handled this manually, by asking the writer for names of the files that she wanted merged; merging them into a local copy of the development branch; resolving merge conflicts and QA errors (errors in DITA source, spelling); and eventually merging into the development branch.

For an Oxygen-using client that uses git on the back end I implemented a custom set of actions using the Oxygen git plugin as my code base that automates all the git actions for users, including creating feature branches, checking for updates, letting users know that their local branch is out of date, and so on.

This was not a trivial effort but it wasn't that hard either. Unfortunately this was a work for hire so I can't share the code, but it was mostly an exercise in figuring out how to automate the git actions and do the necessary error handling, as well as my personal struggle with how to do UI programming in Java and Oxygen (not something I had done much of to that point).

But looking Kris' initial question, I think the key is the amount of time that feature branches go without being updated to reflect the develop branch--doing that frequently will avoid most merge conflicts and limit the ones that do occur to hopefully clear reasons. As long as authors are not simultaneously working on the same topics or maps or inadvertently modifying files they shouldn't (for example, by reformatting the markup in a file that hasn't had its content otherwise changed) there should not be many merge conflicts. But ensuring this requires communication among the team members.

Cheers,

E.

--
Eliot Kimber
http://contrext.com


On 12/18/19, 6:52 AM, "Kristen James Eberlein" <dita-users@groups.io on behalf of kris@...> wrote:






Thank you, Chris, Alastair, and Jean-Noël for your feedback.
I do want to set some additional context here:


* I am familiar with and comfortable with Git, although it is
new to the technical writers on this project. (And perhaps
frightening to the manager.) I started using Git when the
DITA-OT project moved to it from SVN, and I'm still grateful for
all the help that Jarno Elovirta, Robert Anderson, and Roger
Sheen gave me as I learned Git.


* Yes, good practices for Git are just standard, good software
development practices: Keep feature branches discrete; update
feature branches by pulling from the current development
repository frequently; commit thoughtfully and frequently with
good commit messages, so that cherry picking is a possibility.

* For this project, the technical writers are using oXygen, and
we can control formatting through common project files to reduce
conflicts.


That said, version control with Git can have a steep ramp-up for
technical writers. Any thoughts about how to best minimize that?
Useful resources?
And how to best plan a process for the scenario in which a
writer wants to merge some (but not all content) from a feature
branch into the development branch? In the absence of
granular commits with descriptive commit messages, cherry picking
is a not a realistic option.
So far I've handled this manually, by asking the writer for names
of the files that she wanted merged; merging them into a local
copy of the development branch; resolving merge conflicts and QA
errors (errors in DITA source, spelling); and eventually merging
into the development branch.

Best,
Kris

Kristen James Eberlein
Chair, OASIS DITA Technical Committee
Principal consultant, Eberlein Consulting
www.eberleinconsulting.com <http://www.eberleinconsulting.com>
+1 919 622-1501; kriseberlein (skype)

Regarding this:
And how to best plan a process for the scenario in which a writer wants to merge some (but not all content) from a feature branch into the development branch? In the absence of granular commits with descriptive commit messages, cherry picking is a not a realistic option.
===========

My first instinct is to say, just don't.  Ok, that's not an answer.  My second instinct is to slap your wrist for the lack of granular commits and descriptive messages.  I do consider that a viable answer.  Have a meeting and instill good habits!  (if you're worried about merging too many commits into develop at a go, look up squash commit.)

The practice of creating a local feature branch, and then pushing that feature branch out to origin should be your starting point.  Now you have two branches...  One that's local and one that's remote.  As you work on the local, when you reach a milestone you can add/commit the affected files, and then push them out to the remote.  Note that push will only push out committed files.  So you can have a number of files that you have changed on local, plus a number of other files you have changed AND committed on local.  When you push, only the latter set will go out to remote.  At any time, you can merge your remote into develop, because it only has finished files.  So you can use this as a way to stage your work in a way that, WITH PROPER PLANNING AND GOOD HABITS, you can incrementally merge your work into develop.

There is a fly in this ointment.  In all cases, you MUST REGULARLY (at least once a day, plus just before you plan any merge into develop) check out develop, pull, check out your feature branch, then merge develop...  And then push the merged-into feature branch out to remote feature.  Only then will the remote of your feature branch be in sync with ongoing changes in develop.    Here's the problem...  If you have un-committed changes in your local feature branch, git will not allow a merge from develop (for obvious reasons). 

All is not lost...  Git has a workaround.  Before you do the merge, execute git stash. This hides all your un-committed changes.  Then you can merge develop into your feature branch, push, and then execute git stash pop to reveal your hidden files.  This almost covers all your bases.  But still a fly in the ointment...  Stash will not hide a new file.  If you have added a file to your local, and you have not committed that change, stash will not hide it. 

So...  I believe that in nearly all reasonable cases, this can handle your situation.  You need:

• Reasonable planning. I question the need to merge a subset of a feature branch -- maybe you need more than one feature branch in this case?  Are you dividing feature branches by task, or by so-called feature in the documentation?  I suggest the former.  But if you are legitimately working on unfinished changes, and you need to merge in already-finished changes, then judicious use of commit and push should take care of you.
• Clear understanding of Git. Somebody has to step up to be the authority.  And that person needs a contact outside of your team who can help you through goofy issues.
• A local and a remote version of your feature branch.
• Constant, religious attention to merging develop into your local feature branch, and pushing the merge out to your remote feature branch.
• A reasonable and clear concept of WHEN to commit/push from local to remote.  Your goal is to make sure the remote feature branch is always ready to merge into develop.
• Smart management of new files.
• Occasional use of stash.

Another wrinkle to consider...  One reason to push your feature branch to remote is so that other people can see it and modify it.  If that is your plan, then you muse be even more scrupulous.  And you must not only pull develop and merge it into feature, you must also regularly pull remote feature into local feature. 

I'll say that we work this way all the time.  I was first introduced to Git in this project -- we switched from svn.  We have had people on the project with varying git experience...  Some people with zero experience, and not much command-line confidence either.  Neither fire nor poisonous toads have rained from the sky.  Keep your eyes open, and think ahead.  And yes, you might need to bend some ways of thinking or doing if you want to get the most out of this tool.  That's how technology works...

I really appreciate everyone's feedback and advice; it reminds me what a wonderful community we have here! Chris, thanks especially for your most recent e-mail.

Yes, education about Git and Git best practices is critical. It's my current priority to help the team with that.

And to those who are saying "Why hasn't Kris helped them with that already?," I need to say that getting people from 0 to 60 miles/ 100 km a hour is impossible to do overnight. My initial focus was 1) build automation, and 2) ensuring that the DITA source was free of errors. While this is a small help system (1,725 topics), it is loaded with cross references (4,312) and images (1,408). The technical writers badly needed to understand the concept of dependencies, and the cascading effects of deleting or refactoring a topic.

So back to Git and the key issue of technical writers needing to merge some (but not all) of a feature branch into the development branch ... This is a result of how product management operates.

Here is the typical scenario. Product management opens a Jira ticket that focuses on needing to document a new product feature (or change in product architecture) into the documentation. The manager does her best to qualify and negotiate scope, and then assigns the ticket to a technical writer. The technical writer assesses the impact to the existing documentation (and its spaghetti structure); she tries to understand and stage the work as best as possible. The technical writer works hard on improving the content. But approval for deploying the content falls to product management. When they do review the content, they want to tweak words, and so postpone deploying the content for a cycle. Next cycle, they want to cherry pick what is deployed: "I'm good with this; it can go. No, this needs more work; it should wait for another release."

Obviously, this is not a good or sustainable workflow, and changing it is a big priority for the manager. But it's not going to change overnight. Remember that the manager and the technical writers are new to the company and the product. Luckily they are smart and determined.

So I'm still left trying to determine what are the best processes that I can immediately put in place to mitigate pain and churn. I'm leaning towards:

• Better planning for documentation changes that require feature branches. Can the feature branches be more modular? Better planned and scoped?
• Increased education about Git best practices, especially around working with feature branches 
• Team guidelines for guidelines for 1) what go should go into a commit, and 2) commit messages
• Mentoring one of the technical writers as he ramps up to become the team Git expert (already underway)

I'll have to think about whether commiting but not pushing some files and then stashing them is an option. I suspect not, since content in the feature branch needs to pushed in order to generate build artifacts that the stakeholders can download and review.

-- 
Best,
Kris

Kristen James Eberlein
Chair, OASIS DITA Technical Committee
Principal consultant, Eberlein Consulting LLC
www.eberleinconsulting.com
+1 919 622-1501; kriseberlein (skype)

On Thu, Dec 19, 2019 at 05:03 AM, Kristen James Eberlein wrote:

The technical writer works hard on improving the content. But approval for deploying the content falls to product management. When they do review the content, they want to tweak words, and so postpone deploying the content for a cycle. Next cycle, they want to cherry pick what is deployed: "I'm good with this; it can go. No, this needs more work; it should wait for another release."

Yikes...  What a PM nightmare.  Why is PM always so difficult to work with?  What you're trying to do is apply a tool that counts on technical discipline to a workflow that counts on social lack of discipline.  The first thing I would do as a manager is lay out the manual process that is required to support this lack of discipline, and assign a cost to it.  Nothing is free.  If they want to insist on their work flow, then they need to assume the cost in both budget and schedule.  You could expect a combination of them increasing their discipline, and build managers coming up with ways to more easily accommodate their requirements.  I know that doesn't specifically answer your question, but I can't help myself.

Using local/remote versions of the feature branch, you could just hold off on commits until you have the OK from PM.  That plus stash will work up until you need to add new files. 

Another thing that might help could be smaller feature branches.  How is it possible, for a given "feature", that some changes are good to go and others are not?  Are these all changes about the same thing?  In what world is it ok to correct half of a feature, and leave the other half incorrect?  I would explore whether their concept of a "feature" is the same as yours, and see if you can adjust the chunking of your feature branches.

Not coming up with much more...  You need to brainstorm on this.  But the key feature I see here is that it is not strictly a technical problem.  I think there's a social aspect as well.  I'm a naturally obnoxious person, and so I would shine some light on the social problems.  Not clear that you have that option...

You've nicely summed up the fundamental problem: "What you're trying to do is apply a tool that counts on technical discipline to a workflow that counts on social lack of discipline."

More background: The product managers are not software engineers; they are marketing guys.

Yes, this is not just a technical problem in search of a solution. Change management is definitely required, and I certainly can clarify for the tech doc manager that "the problem" is not DITA or Git; a big component of "the problem" is the product managers' workflow.

As a consultant, I cannot be obnoxious. I think most people would describe me as outspoken and persistent.

What you are pointing out is basically rewriting the history of feature
branches, which is totally against Git's way of doing in its standard
usage. In this case, there's need for more advanced git management, with
all the required caveats.

So first of all, if we are to rewrite history, it's mandatory to not
introduce merge commits. Basically that means rebasing feature branches.
Be careful to make people understand that these rebased feature branches
can not be modified by more than one writer.

Then, when it's time to cherry pick the changes, I would put on the
"expert mode":
 

* Checkout the branch and rebase it on top of develop
* Reset the working copy to develop. The history seems to be at
develop, but the working copy holds all the changes of the branch.
As a result, all the changes of the branch are available in the
working copy for selection of hunks to add to the next and unique
commit.
* Stage changes that go in the commit (for later merge into develop)
hunk by hunk,effectively leaving in the working copy changes that
won't make it into develop.
* Commit the changes to be merged, stash the rest
* Check out develop, spawn a new branch, unstash and commit as the
branch "not merged".
* Merge the first branch into develop.


This is quite involved, but that's a minimum to rewrite history.

JN