Docker containers for building deviation
- mwm
- Topic Author
- Offline
See the description of mwmeyer/deviation description for some suggested uses for these. But the short version is that you'll want to run containers that mount your local source tree as a volume and build against that, as they don't include sources. This means the images never change.
There are four of them:
- mwmeyer/deviation - has the tools recommended in the deviation README to build transmitter images and zips of the same.
- mwmeyer/deviation-emulators - this adds the tools needed to build the windows emulators and zip files. The command "make release" in this container will build everything you'd want to upload.
- mwmeyer/deviation-manual - This only has tools for building the manuals.
- mwmeyer/deviation-sources - This is a data-only image that comes pre-populated with the sources for deviation and the manual. You probably don't want this unless you're going to be building in the cloud.
This is a different type of solution than the existing virtualbox image. Which you'd prefer will depend a lot on you and your preferred tools.
Do not ask me questions via PM. Ask in the forums, where I'll answer if I can.
My remotely piloted vehicle ("drone") is a yacht.
Please Log in or Create an account to join the conversation.
- Antfurn
- Offline
- Posts: 10
Please Log in or Create an account to join the conversation.
- silpstream
- Offline
- Posts: 244
Thanks very much!
Please Log in or Create an account to join the conversation.
- mwm
- Topic Author
- Offline
The one odd thing is the "deviate" script, which was designed as a simple "build a branch" command. You can safely ignore it if yu want.
As for the dfuse tool, they can be used to build those. You'll want to mount the sources on the container, and you'll have to make sure the libopencm3 sources as well as the bootloader are in it. The toolchain is the same one, though.
Do not ask me questions via PM. Ask in the forums, where I'll answer if I can.
My remotely piloted vehicle ("drone") is a yacht.
Please Log in or Create an account to join the conversation.
- boblethai
- Offline
- Posts: 1
I'm quite new in docker (and deviation as well, I must admit!)
The given command does not work for me:
[root@budapest Deviation_Build_Docker]# docker run deviation -v ~/src/deviation:/deviation --name deviation mwmeyer/deviation /bin/bash
Unable to find image 'deviation:latest' locally
Trying to pull repository docker.io/library/deviation ... not found
Error: image library/deviation:latest not found
but if I do:
[root@budapest ~]# docker run --rm -i -t -v ~/src/deviation:/mnt mwmeyer/deviation /bin/bash
root@e4eada67c324:/deviation/src#
it looks better.
My docker version is:
[root@budapest ~]# docker --version
Docker version 1.8.2-el7.centos, build a01dc02/1.8.2
probably not the most up to date... because I'm running centos 7.
Bob
Please Log in or Create an account to join the conversation.
- mwm
- Topic Author
- Offline
However, your first command line should start "docker run -v ...", not "docker run deviation -v ...". That extra "deviation" becomes the image name, which makes it search for it in the docker.io/library instead of my account, hence the error message.
I've fixed the description to be correct (which means add "-it" as well). Thanks for reporting it.
Do not ask me questions via PM. Ask in the forums, where I'll answer if I can.
My remotely piloted vehicle ("drone") is a yacht.
Please Log in or Create an account to join the conversation.
- silpstream
- Offline
- Posts: 244
I used Kitematic to link the volume and start up the container and then went to the cli ("docker cli" button on the bottom left of Kitematic) and did a "docker attach deviation" to get inside.
I then ran "make zip TARGET=devoXX" (replace XX with model number) from the src folder. This created a zip file with everything I needed to flash using the 0.8 version of uploader tool. Remember to run "make distclean" after to clean things up for the next build.
Please Log in or Create an account to join the conversation.
- PhracturedBlue
- Offline
- Posts: 4402
hub.docker.com/r/deviationtx/deviation-docker/
The docker-file and build script can be found here:
github.com/DeviationTX/deviation-docker
1st pull the image:
docker pull deviationtx/deviation-docker
next make a directory where you want the builds to go on your host machine:
mkdir ~/devo_builds
now you can start the image in a couple ways:
As a developer where you already have a cloned deviation repo
docker run -it -v <path to git repo head>:/git -v ~/devo_builds:/release --name deviation_build deviationtx/deviation-docker
NOTE: <git repo head> is the directory above your repo (it should contain a subdirectory called 'deviation')
As a user without a cloned repo:
docker run -it -v ~/devo_builds:/release --name deviation_build deviationtx/deviation-docker
On subsequent runs, you can do:
docker start -i deviation_build
The scripts will 1st check whether there have been any updates to the build script recently. It will then update your deviation repo (or clone it if you didn't provide one). next it will show a menu that will let you choose what you want to build. Third it will fetch any additional files needed to run the build. Lastly it will build the selected targets and store the results in the ~/devo_builds dir on your host machine.
The build env will be cached, so the next time you run, it will be much faster.
The build script is still very crude. I will provide options to further configure the builds in the future. I also need to test building on Windows, and write some instructions, but my understanding is that I should be able to build a lightweight installer to automate all of that.
If this works, I plan to make it the only supported build system. This image should be nearly identical to the version used by our official and nightly builds, so there should be no cases where a build works in one place and not in another.
The primary downside is that the file image is quite large.
I'd like any feedback if you are successful with this method.
Please Log in or Create an account to join the conversation.
- silpstream
- Offline
- Posts: 244
Very nice. Having the interface will go a long way to easing the build process. Especially if you are going to start adding in build time options later on.PhracturedBlue wrote: now you can start the image in a couple ways:
As a developer where you already have a cloned deviation repo
docker run -it -v <path to git repo head>:/git -v ~/devo_builds:/release --name deviation_build deviationtx/deviation-docker
NOTE: <git repo head> is the directory above your repo (it should contain a subdirectory called 'deviation')
As a user without a cloned repo:
docker run -it -v ~/devo_builds:/release --name deviation_build deviationtx/deviation-docker
On subsequent runs, you can do:
docker start -i deviation
Worked for me on "devo7e". "devo7e-256" throws up an error "No rule to make target `zip_devo7e-256'".
You have a typo error for subsequent runs above, should read as "docker start -i deviation_build", if not mwm's docker container gets started instead.
I've not looked into the scripts you have.
I'm wondering, for those of us developing on forked clones (most developers would be right?), I assume that it doesn't actually do any updating and will not switch me back to master branch. Developer forks are probably not a use case right?
Thanks for this!
Please Log in or Create an account to join the conversation.
- PhracturedBlue
- Offline
- Posts: 4402
The current proff of concept GUI is using 'dialog' which is very limited in its capability. I found something called 'snack' for python which allows building more complicated text-based GUIs. I'm going to rework the current script into a python script which will allow me to add some developer-centric options. I will take a look at the devo7e-256. Not sure why that wouldn't work.
Please Log in or Create an account to join the conversation.
- PhracturedBlue
- Offline
- Posts: 4402
Please Log in or Create an account to join the conversation.
- PhracturedBlue
- Offline
- Posts: 4402
The issue I have is that I want the 'Update GIT' button to run a pull and return to the menu. This works fine on my linux machine, but for some reason in docker it hangs. I've temporarily changed the code to exit after an update to avoid the hang, which isn't ideal, but it will do for the time being until I figure out what is wrong in docker.
Please Log in or Create an account to join the conversation.
- PhracturedBlue
- Offline
- Posts: 4402
Windows steps are nearly identical to linux:
Install docker for windows. You must have a 64-bit windows installation of a supported Windows variant:
Instructions here: docs.docker.com/engine/installation/windows/
If you already have VirtualBox installed, make sure it is at the latest version before you begin AND unselect it from the list during docker install. If you don't have the latest VirtualBox, the install will pass, but initialization will fail. It took me a while to figure out what was going on
now click 'Docker Quickstart Terminal'
When you eventually get a prompt (this is a one-time setup), run:
docker pull deviationtx/deviation-docker
And then:
mkdir /c/Users/<username>/Release
now:
docker run -it --name deviation_build -v /c/Users/<username>/Release:/release deviationtx/deviation-docker
and next time:
docker start -i deviation_build
Now the question is: Is this too much work to be worth it? Is there a better solution, or a way to simplify the process?
Please Log in or Create an account to join the conversation.
- mwm
- Topic Author
- Offline
I guess the question is - how hard is this compared to installing a build environment "by hand". For the Mac, I believe it's much easier, but never bothered to do that as we don't have instructions for it. For windows, it seems a bit easier, but again - I haven't done that. For linux, it fixes the issue of making sure you have the right build chain, which has gotten a bit painful to deal with.
Are these 32-bit or 64 bit Linux images? The ones I built are 64 bits and can't work on my BSD desktop.
Do not ask me questions via PM. Ask in the forums, where I'll answer if I can.
My remotely piloted vehicle ("drone") is a yacht.
Please Log in or Create an account to join the conversation.
- sbstnp
- Offline
- Posts: 649
So, try with docker run [...] --net=host [...], should allow git to fetch changes.
Devo 10 + 4in1
FrSky Taranis + TBS Crossfire
Please Log in or Create an account to join the conversation.
- PhracturedBlue
- Offline
- Posts: 4402
I think all docker images are by default 64-bit. I am not sure how to even change it. Why does BSD not support 64bit images?
Please Log in or Create an account to join the conversation.
- richardclli
- Offline
- Posts: 199
Please Log in or Create an account to join the conversation.
- silpstream
- Offline
- Posts: 244
1) I cannot update from git, I'm guessing it might be because I'm using my own repository and am using ssh keys, maybe?
Updating build.py
Warning: Permanently added the RSA host key for IP address '192.30.252.128' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
3) Could we possibly also specify the git branch prior to pull. Makes life easier when testing code in a branch. Of course if (2) comes true, then we should be okay.
Thanks again!
Please Log in or Create an account to join the conversation.
- PhracturedBlue
- Offline
- Posts: 4402
Please Log in or Create an account to join the conversation.
- silpstream
- Offline
- Posts: 244
Please Log in or Create an account to join the conversation.
- Home
- Forum
- Development
- Development
- Docker containers for building deviation