Docker containers for building deviation

More
29 Apr 2016 00:35 #47395 by hexfet
Replied by hexfet on topic Docker containers for building deviation
Yep, some of the files being written to were owned by root from previous builds. Used chown as a safer alternative to rm -rf :)

Please Log in or Create an account to join the conversation.

More
29 Apr 2016 02:57 - 29 Apr 2016 02:57 #47399 by PhracturedBlue
Replied by PhracturedBlue on topic Docker containers for building deviation
Would someone with a Mac be willing to try to use the latest docker image, mounting a pre-checked out git repo from outside the image?

Assuming that works, I think I'm willing to say this is the official way to build Deviation going forward.
Last edit: 29 Apr 2016 02:57 by PhracturedBlue.

Please Log in or Create an account to join the conversation.

More
29 Apr 2016 16:46 - 29 Apr 2016 16:48 #47433 by silpstream
Replied by silpstream on topic Docker containers for building deviation
Connection was really slow for some reason and it took me over 2 hours to complete the pull.

Tested all builds (except x9d related ones) on OS X 10.10.5 against my forked git repository. Everything even the emulators work great. Nothing (not even permissions or owners) needed changing and worked on the first pass.

Great stuff PB! Thanks so much for this! :woohoo:

On a side note: it doesn't go back to the gui after building. This is intended, right?
Last edit: 29 Apr 2016 16:48 by silpstream.

Please Log in or Create an account to join the conversation.

More
29 Apr 2016 19:56 #47440 by PhracturedBlue
Replied by PhracturedBlue on topic Docker containers for building deviation
Yes that is intended.

Thanks everyone for your help validating this. I'll start working on a proper cheatsheet documenting this process

Please Log in or Create an account to join the conversation.

More
30 Apr 2016 15:22 #47493 by PhracturedBlue
Replied by PhracturedBlue on topic Docker containers for building deviation
I have tried to fully document the Docker build procedure on the wiki here:
www.deviationtx.com/wiki/development/docker

And I have changed the repo README to reflect the now recommended build procedure

If anyone finds any issues please let me know (or even better just fix them yourself)

Please Log in or Create an account to join the conversation.

More
30 Apr 2016 18:31 #47500 by TheSFReader
Replied by TheSFReader on topic Docker containers for building deviation
Well done !!! :)

Please Log in or Create an account to join the conversation.

More
30 Apr 2016 19:16 #47507 by HappyHarry
Replied by HappyHarry on topic Docker containers for building deviation
awesome work, this has seriously lowered the bar for people wanting to jump in and build their own deviation images. it also makes things easier for those with a little bit more knowledge as no more worrying about keeping gcc-arm and it's requirements from updating etc, thanks :)

Please Log in or Create an account to join the conversation.

More
30 Apr 2016 19:55 #47509 by silpstream
Replied by silpstream on topic Docker containers for building deviation
Nice work! Thanks v much!

Please Log in or Create an account to join the conversation.

More
01 May 2016 19:52 - 01 May 2016 20:01 #47563 by hexfet
Replied by hexfet on topic Docker containers for building deviation
Agree it's a nice advance in providing a standard build environment. Thanks PB! On my wish list...

1) Is it possible to support building deviation-manual? I'm not sure what's necessary - make currently fails right away since virtualenv is not installed.

2) Would be nice to be able to make the container a build server - send it a make command and it runs it. Currently using it to build while developing means switching to the container for building, then back to native for editing and debug. I ended up installing the build environment locally. Likely in the category of "PR welcome" :)
Last edit: 01 May 2016 20:01 by hexfet.

Please Log in or Create an account to join the conversation.

More
01 May 2016 20:26 #47566 by PhracturedBlue
Replied by PhracturedBlue on topic Docker containers for building deviation
1) I was considering adding manual build support for HTML, you only need virtualenv. For pdf, you need virtualenv and imagemagick. It should be quite easy to add
2) I agree this would be helpful. As a developer, I found that the interactive mode is too slow. I really need to be able to run '<some command> <make options>' for this to be efficient. You can pass env variables to docker when creating containers, but you can't change them when doing a 'docker start'. There is a cludgey solution of saving the command to the shared file system and then parsing that on startup, but it is pretty nasty. I'd like to find a solution to this one.

Please Log in or Create an account to join the conversation.

More
11 May 2016 04:07 #48186 by PhracturedBlue
Replied by PhracturedBlue on topic Docker containers for building deviation
I've now added a mechanism to run arbitrary commands while starting the docker image.

You need to run 'docker start' at least once in order to fetch the latest script. This is only useful if using an external git volume, and cannot be used if you didn't mount git during the 'docker create' step.

in the deviation git repo is a script called 'dmake'. If you call this (I recommend adding it to your path) instead of 'make' it will launch the build using the docker image.

For instance 'dmake devo10' will use docker to build the devo10 image. Note that the resulting dfu is not copied to the results directory. Doing so would be complicated, but it should be sitting in the src dir of your git repo, so all is not lost. This feature is only intended for developers in a hope to make the docker image useful for development. Note that this does not do any of the initialization that is done through the gui. So if any software installation is needed, you must do that using the docker gui 1st (for instance to setup the windows emu env).

This is all very un-docker-like, but why bother having guidelines if I can't flaunt them?

Please Log in or Create an account to join the conversation.

More
11 May 2016 09:25 #48192 by Richard96816
Replied by Richard96816 on topic Docker containers for building deviation
dmake is very cool.

Please Log in or Create an account to join the conversation.

More
11 May 2016 19:44 #48227 by HappyHarry
Replied by HappyHarry on topic Docker containers for building deviation
using this method to build the actual tx binaries works perfectly fine, and this is probably outside the use case but compiling the emulators are falling over

here's the output for emu_devo7e
phil@ubuntu:~$ dmake emu_devo7e
Skipping update of build.py
 + Compiling 'target/common/emu/fltk.cpp'
target/common/emu/fltk.cpp:28:19: fatal error: FL/Fl.H: No such file or directory
 #include <FL/Fl.H>
                   ^
compilation terminated.
make: *** [objs/emu_devo7e/fltk.o] Error 1
phil@ubuntu:~$ 

and the same for emu_devo10
phil@ubuntu:~$ dmake emu_devo10
Skipping update of build.py

SNIP

 + Compiling 'target/common/emu/fltk.cpp'
target/common/emu/fltk.cpp:28:19: fatal error: FL/Fl.H: No such file or directory
 #include <FL/Fl.H>
                   ^
compilation terminated.
make: *** [objs/emu_devo10/fltk.o] Error 1
phil@ubuntu:~$ 

it's the same if you try emuzips
phil@ubuntu:~/git/deviation$ dmake emuzips
Skipping update of build.py

SNIP

 + Compiling 'target/common/emu/fltk.cpp'
target/common/emu/fltk.cpp:28:19: fatal error: FL/Fl.H: No such file or directory
 #include <FL/Fl.H>
                   ^
compilation terminated.
make[1]: *** [objs/emu_devo6/fltk.o] Error 1
make: *** [zip_emu_devo6] Error 2

i'm probably doing something wrong though lol

Please Log in or Create an account to join the conversation.

More
11 May 2016 20:18 #48229 by PhracturedBlue
Replied by PhracturedBlue on topic Docker containers for building deviation
I did not include the ability to build linux-based emulators in the script yet.
It can be done by going into the shell and doing 'sudo apt-get install libfltk1.3-dev' (this is mentioned in this thread above)

I'll add that sometime soon (since I use the capability myself).

If you were actually looking to build the windows emulator (whcih does run fine in Wine by the way), you instead want:
'make win_emu_devo7e' or 'make zip_win_emu_devo7e'

Please Log in or Create an account to join the conversation.

More
11 May 2016 20:56 #48231 by HappyHarry
Replied by HappyHarry on topic Docker containers for building deviation
thanks for the help pb, yeah it was the linux emu's i was after and installing fltk did the trick, i needed to install it inside the container as you said to build it, but also in the main os to be able to run them

Please Log in or Create an account to join the conversation.

More
12 May 2016 00:26 #48241 by hexfet
Replied by hexfet on topic Docker containers for building deviation
Thanks! dmake works well

Please Log in or Create an account to join the conversation.

More
12 May 2016 01:22 - 12 May 2016 01:48 #48242 by PhracturedBlue
Replied by PhracturedBlue on topic Docker containers for building deviation
I've now added the ability to build the linux emulator.
From the shell:
sudo /root/build.py --linux-prereq

then you can use 'dmake emu_devo7e'

I've also added the ability to build the manual through the docker image.
From the shell:
sudo /root/build.py --manual-prereq

You must clone the repo into the same top-level dir you used for deviation (so if you used ~/git/deviation, you would use ~/git/deviation-manual). you do not need to re-create the image since we mount the '~/git' dir
then you can then do:
dmake TARGET=devo10 pdf

If you've previously built the manual locally, make sure to do 'make clean' first. Also both of the 'sudo' commands above will need to download a significant amount of stuff so be prepared for that.

Note also that I've updated the 'dmake' script so that (a) you can move it to someplace other than the deviation git dir, and (b) it can build other repos (like the manual). The manual doe not have its own copy of dmake.
Edit: Also, dmake now requires you to be in the same dir as the Makefile (just like normal 'make')

Lastly, I have not added ether the linux-emu or the manual builds to the gui yet. I'll get there eventually.

I have now added the manual build instructions to the docker wiki page.
Last edit: 12 May 2016 01:48 by PhracturedBlue.

Please Log in or Create an account to join the conversation.

More
16 May 2016 18:50 #48538 by FDR
The docker menu is a bit weird on my Windows: B)

Attachments:

Please Log in or Create an account to join the conversation.

More
16 May 2016 19:29 #48540 by HappyHarry
Replied by HappyHarry on topic Docker containers for building deviation
yeah mine is the same as yours, but it's been like that from the start

windows


linux

Please Log in or Create an account to join the conversation.

More
16 May 2016 19:29 #48541 by PhracturedBlue
Replied by PhracturedBlue on topic Docker containers for building deviation
yes that is how it is. I think it is because of how the TERM variable is set. I tired other options for cygwin and mingw and the results were worse. At least the current solution is usable. Apparently there is a new 'Docker for Windows' that is in Beta that uses an improved VM system (no more virtualbox). It may help with this too. I haven't been able to test it yet.

Please Log in or Create an account to join the conversation.

Time to create page: 0.092 seconds
Powered by Kunena Forum