I am presenting at the CLA Summit in Krakow. As soon as I get the video of the presentation (if there is one), I will add it to this post.
This time I am focusing more on the different approaches to organizing a large project in LabVIEW. My personal preference is to use project libraries and the approaches that I present can apply not only to DQMH projects or Actor Framework projects but to any LabVIEW project that is organized using LabVIEW Project Libraries or LabVIEW classes.
The demonstrations are the same I intended to use for my presentation at GDevCon1, but that I did not have time to show in more detail there.
Using SubModules, SubRepositories or SVN externals to organize your libraries
What I am proposing is for you to create a repository for each project library that will be used in multiple repositories. You might also choose to follow this approach so different LabVIEW programmers can work in different repositories. Then you can have the project library be in its own repository and have the project repository load it as a submodule. You can clone or download the example at github.com/DelacorLLC/GDevCon1_CML-DQMH.git and follow along with the demonstration video below where I show you how to add the Acquisition DQMH module as a subrepository of Libraries, which it is in itself a subrepository of CML-DQMH repository.
The video below shows how it looks like to clone a repository that has sub-repositories. Note how the sub-repositories are specified at a specific revision. This means the repositories themselves can be at a different revision than the one the main repository is pointing to. Also, note how the name of the repository can have a different name than the folder you clone it to.
If you would like for us to discuss how to do the same with Mercurial (Hg) or SVN, let us know via the comments and we will cover this in a future blog post.
Using JKI VIPM packages to share project libraries
The same code above is built using packages, where each DQMH module is saved in its own submodule (or subrepository) and built into packages. The calling code calls the PACKAGED code, not the original code.
You can find an example at github.com/DelacorLLC/GDevCon1_VIPMDemo
Package DQMH Module as PPL
Edit [February 7th, 2020]: If you are having problems when you try to use DQMH built as a PPL, there is a bug in LabVIEW (BUG ID 968709) that doesn’t rename the namespace of the typedef inside the Queue refnum inside the private data of the class. The workaround is to make sure you don’t remove the block diagram of the enqueue VIs when building the PPL.
Matthias Baudot from StudioBods, another DQMH Trusted Advisor, created a video that illustrates the steps to package a DQMH module into a PPL. He presented about this and a lot more about PPLs during his presentation at NI Week 2018 called Use PPLs to Build and Deploy Large, Scalable Applications and Get a Head Start on GLLs in LabVIEW NXG. The video below starts at the point (37:05) where he showed this demonstration.
You can find an example at github.com/DelacorLLC/GDevCon1_PPLDemo
Hi Fabiola,
Very nice videos, thank you for sharing your knowledge!
I would be very interested in seeing how you set Subrepositories with Mercurial ! It would be a great for me !
Hi Cyril,
Glad you liked the videos. I will work on a video on how to do it for Mercurial.
Regards,
Fab
Hi Fab,
THanks for your videos.
One question about submodules. What’s the workflow to “work on” the submodule with the following use case?
I start a new project.
I identify one part as reusable library.
I create a specific repository for this library and add it as a submodule in my project.
There is a lot of chance that I’ll develop both project and Library “simultaneously”.
I feel (as far as the module refer to a specific version) that I have to commit changes only on the Library repository and update the project repository to reflect changes. If I work in a team I need to have strong guideline about how to modify this library. Do you have any advice about that ?
Note that with svn:externals I’m used to work more as if I have to working copies nested… It’s seems easier…
Thank your for your hint.
Hi Olivier,
When you add a submodule, the main repository is pointing to a specific version in the submodule repository.
The calling repository won’t point to a new version until you commit changes to the calling repository and it reports that there are changes in the subrepositories.
My suggestion to teams is to tag the “released” versions of the subrepositories so the calling repo is pointing only to released versions.
If you use Gitflow or Hgflow, you can have the calling repository point to the released branch. The subrepositories would have work in progress either in feature branches or in the development branch.
When you pull a new released version of the subrepository, you can check if the code in the calling repository is ready for that new released version, if it is, you commit the calling repository so it points now to that version.
Alternatively, you could have a document like the vipc file in VIPM that would specifically point to which submodules revisions the main repository is pointing to.
Let me know if this makes sense.
Regards,
Fab
Hi Fab,
Your reply makes sense from what I could have read other websites. I’ve been using Gitflow since I added submodules to my project. I think I got the global idea. A subsidiary question would be how you deal with the submodule code when it’s called directly in your LabVIEW project to prevent any changes in the calling repository ? I find that you can set an option in .gitmodules file to ignore modification (https://git-scm.com/docs/gitmodules) but still wondering to know the best way to share the information with the whole dev team.
Anyway thank you for your elaborated reply!
Hi Olivier,
If the team needs to make changes to the submodule, they would have to branch out the submodule to a feature branch and implement the changes and then go through the regular pull request flow to get their changes into the release branch. Basically, treat the change as you would treat any other changes in other projects, whether you follow GitFlow or not. Once the change goes through the regular release process, it can be called from the calling repository.
Does this answer your question?
Regards,
Fab
Yes, it does.
Thanks.
PS: Completly off topic I din’t receive notification of your reply… That’s why I’m late to reply too 😉
Edited the article to include this about issues when building PPLs:
Edit [February 7th, 2020]: If you are having problems when you try to use DQMH built as a PPL, there is a bug in LabVIEW (BUG ID 968709) that doesn’t rename the namespace of the typedef inside the Queue refnum inside the private data of the class. The workaround is to make sure you don’t remove the block diagram of the enqueue VIs when building the PPL.
Hi,
Your GDevCon1_CML-DQMH Public Repository for CML DQMH demo presented at GDevCon1 has a “mid-level” git submodule named Libraries. What is the benefit of a Libraries repo over simply having a folder named Libraries in the top-level repo GDevCon1_CML-DQMH? I like it but I would like to convince others in my department. Thanks. -cc
Hi Chris,
The purpose of that presentation was to show 3 different approaches to modularizing a project in terms of reusable code or code that multiple developers are working on.
My preferred way is to have different repositories for the different modules and build them in VIPMpackages. In that case, the sub repositories serve as a map of where the source code for the VIPM packages is.
There are others who prefer not to use VIPM and manage everything directly in Git, in that case, the folder structure for the original project is kept and the subrepositories are cloned into the Libraries folder.
The reason I prefer the VIPM packages is that there is a clear distinction between released code that has been tested and published to VIPM and is easy to revert to an older version if the new version introduces a bug. If something needs to change, there is a commit history just for that part of the code and it doesn’t get lost in the story of the main project. Also, as code starts to be more stable, those areas of the code are not touched as often and serve as a good measure of how much the code is changing.
All of this could be achieved with sub-repositories without building into packages but it requires more discipline and knowledge of Git to ensure the top-level code is always pointing to the correct revision of the sub-repositories. With VIPM, one can keep a vipc with the list of packages and their versions. With the new announcement from NI, vipc will be a feature that is included in VIPM and you won’t require a pro version for it.
I hope this answers your question but if my answer created more questions, please do not hesitate to contact me here or via the DQMH forum.
Best regards,
Fab