New Docker Files Available! Asking for help with releases

[Matthew-Beckett]

Hi All!

I am new to the project and have forked the Docker images left unmaintained at juliusloman/netxms-dockerfiles on GitHub

I have been manually handling the publishing of the most recent build 3.8.314, these are published and available in GitLab. However as mentioned, right now this is manual. Ideally I foresee a process in which I can run a GitHub Action to check for NetXMS releases and create a pull request with the relevant changes in the Dockerfiles ready for me to merge and release as a new version.

The issue I face is that I am unable to determine a reliable (and simple) mechanism for determining when new releases are made available. My initial plan was to use GitHub however, the repository is only tagged to the major version and I plan to release all revisions that are made available in apt, these can be as specific as X.X.XXX, furthermore, I do not wish to re-compile within the container and instead will pull from the aptitude repository for simplicity and dependability.

I did also attempt RSS via the forum, however the releases are cluttered with posts other than just new releases making parsing a potential failure point in the event of a feed/post layout/content change.

Is there potential within the project to discuss using the GitHub Releases functionality in order to provide me with a feed with which to receive all updates including minor updates in a feed with which I can parse programmatically and begin automating the creation and publishing of Docker Images for NetXMS.

One final note, I have not updates the management console Docker image as it seemed a bit unnecessary, but I am open to supporting this again in future should people have a requirement for it.

Thanks!
Matt

[alk]

Hello.

You can parse https://github.com/netxms/changelog/blob/master/ChangeLog. It's updated before the official release is announced, but once X.Y-CURRENT is changed to full version X.Y.Z - it's frozen.

[Matthew-Beckett]

Brilliant, I'll get onto that tonight. Hopefully it'll be fully automated by the end of next week!

[Matthew-Beckett]

@alk said in New Docker Files Available! Asking for help with releases:

Hello.

You can parse https://github.com/netxms/changelog/blob/master/ChangeLog. It's updated before the official release is announced, but once X.Y-CURRENT is changed to full version X.Y.Z - it's frozen.

Hey,

I am working on this at the moment, and parsing the changelog file is going to be unreliable due to a few inconsistencies in formatting throughout the file. We've managed to workaround the majority of the formatting issues, however some will require a PR to fix.

I will send this over when I have finished the parser. However, looking to the future, it would be really helpful if we could look at one of the following improvements:

• Tagging the netxms/netxms repository as specific at minor version
    Currently it's 3.8, I would need 3.8.314 to be an available tag in order to target it in apt
• Creating GitHub releases for all versions available in apt
    Basically the same reason as above, however you can setup a webhook in this case to trigger my bot (for now it will be on a schedule)

Let me know what you think

[victor]

Hello,

I think we can use lightweight tags for marking release versions. Also, you can use this simple script to switch to specific X.Y.Z version within source tree: https://github.com/netxms/netxms/blob/master/tools/switch_to_version.sh

Best regards,
Victor

[Matthew-Beckett]

@victor said in New Docker Files Available! Asking for help with releases:

Hello,

I think we can use lightweight tags for marking release versions. Also, you can use this simple script to switch to specific X.Y.Z version within source tree: https://github.com/netxms/netxms/blob/master/tools/switch_to_version.sh

Best regards,
Victor

Hi Victor,

My source of truth for packages is APT, I am not building from source inside the container. Currently I am struggling with finding a reliable mechanism to detect new releases from yourselves, unfortunately benig able to switch to a specific build within git doesn't help and also depends on me knowing what version number to switch to.

In my pipeline I have no context, I have to be able to pull a list of versions (that are available in apt) and compare that with my existing releases.

UPDATE:

I've checked out and tested your script and I can understand how versions numbers are now generated. How do you decide to cut a release? Is it not possible to increase the tagging to include releases present in apt?

I don't even need the commit associated with it, just basically an API I can query and get back a list of available NetXMS releases.

[victor]

We decide that we do official packaging when planned features and fixes are ready. As third number in version is generated automatically from git commits it cannot be predicted. i think that lightweight tags is a way to go, we will just put lightweight tag whenever we do deb publishing. We can agree on certain format for those tags so you will be able to distinguish between deb releases and other tags.

Best regards,
Victor

[Matthew-Beckett]

@victor said in New Docker Files Available! Asking for help with releases:

We decide that we do official packaging when planned features and fixes are ready. As third number in version is generated automatically from git commits it cannot be predicted. i think that lightweight tags is a way to go, we will just put lightweight tag whenever we do deb publishing. We can agree on certain format for those tags so you will be able to distinguish between deb releases and other tags.

Best regards,
Victor

Hi Victor,

Brilliant! This is perfect, does that mean starting with the next 3.8.XXX release I can expect to find a tag at netxms/netxms with that new version as it's name?

[victor]

Yes, from now on we will put lightweight tags on each release. We will also experiment with GitHub releases, will see how it will work.

Regarding maintenance and documentation - we are ok to include Docker based installation into installation guide. Do you mind moving source files for Docker deployment into separate repository under netxms project on GitHub?

Best regards,
Victor

[Matthew-Beckett]

@victor said in New Docker Files Available! Asking for help with releases:

Yes, from now on we will put lightweight tags on each release. We will also experiment with GitHub releases, will see how it will work.

Regarding maintenance and documentation - we are ok to include Docker based installation into installation guide. Do you mind moving source files for Docker deployment into separate repository under netxms project on GitHub?

Best regards,
Victor

Yes, there is potential to bring Matthew-Beckett/netxms-dockerfiles into netxms/dockerfiles. Though I would prefer we look into this once the images are more mature and stable, currently I depend on GitLab CI for building and publishing the images, if we were to bring the Dockerfiles into a repository under NetXMS I would like to move this to GitHub Actions.

[victor]

We just published new release 3.8.366. I have added tag release-3.8.366 as well as GitHub release.

Best regards,
Victor

[Matthew-Beckett]

@victor said in New Docker Files Available! Asking for help with releases:

We just published new release 3.8.366. I have added tag release-3.8.366 as well as GitHub release.

Best regards,
Victor

Hey Victor,

Thanks so much this, it makes my job handling automated releases much easier. I am currently in the process of scoping out the most efficient method of handling release automation. Currently the short list contains

• GitHub Bot
• GitHub Action Scheduled
• GitLab CI Job Scheduled

I am leaning towards the GitHub action, I suspect it will end up that the action runs in the context of Matthew-Beckett/netxms-dockerfiles and will need a webhook from netxms/netxms to trigger the pipeline on creation of a Release.