| *** e0ne has joined #openstack-loci | 04:59 | |
| *** e0ne has quit IRC | 04:59 | |
| *** ispp has joined #openstack-loci | 07:12 | |
| *** ispp has quit IRC | 07:20 | |
| *** ispp has joined #openstack-loci | 07:21 | |
| *** ispp has quit IRC | 07:46 | |
| *** ispp has joined #openstack-loci | 08:18 | |
| *** e0ne has joined #openstack-loci | 08:55 | |
| *** e0ne has quit IRC | 09:05 | |
| *** e0ne has joined #openstack-loci | 09:13 | |
| *** pbourke has quit IRC | 09:31 | |
| *** pbourke has joined #openstack-loci | 09:32 | |
| *** ispp has quit IRC | 10:20 | |
| *** ispp has joined #openstack-loci | 10:22 | |
| *** ispp has quit IRC | 10:25 | |
| *** e0ne has quit IRC | 10:28 | |
| *** e0ne has joined #openstack-loci | 10:34 | |
| *** hrw has joined #openstack-loci | 10:58 | |
| hrw | hi | 10:58 |
|---|---|---|
| hrw | did someone tried to build loci on non-x86 system? | 10:59 |
| *** e0ne has quit IRC | 11:02 | |
| *** ispp has joined #openstack-loci | 11:15 | |
| *** e0ne has joined #openstack-loci | 11:15 | |
| *** gema has joined #openstack-loci | 11:20 | |
| gema | SamYaple: ping | 11:20 |
| gema | SamYaple: we are going to start adding support for Arm64 to LOCI if you don't mind :D | 11:20 |
| gema | SamYaple: hrw raised a bug: https://bugs.launchpad.net/openstack-loci/+bug/1781905 | 11:20 |
| openstack | Launchpad bug 1781905 in LOCI "Does not work on !x86 architectures" [Undecided,New] | 11:20 |
| hrw | SamYaple: o/ | 11:21 |
| gema | SamYaple: anyway, the background for this work is that we want to enable openstack-helm | 11:21 |
| gema | and be ready for when/if kolla decides to move to them | 11:21 |
| *** ispp has quit IRC | 11:33 | |
| *** ispp has joined #openstack-loci | 11:35 | |
| hrw | now have to find out why pip is called in parallel when 'requirements' image is built | 12:01 |
| hrw | as on non-x86 we need to build some modules and numpy takes huge amount of time | 12:02 |
| hrw | in meantime some other are attempted to install and fail as there is no numpy (yet) | 12:02 |
| *** ispp has quit IRC | 12:13 | |
| hrw | ha! CASS_DRIVER_BUILD_CONCURRENCY=8 | 12:15 |
| *** openstackgerrit has joined #openstack-loci | 12:19 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/loci master: bindep: we need libjpeg to build Pillow https://review.openstack.org/582935 | 12:19 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/loci master: pip_install: disable '--only-binary' for !x86-64 architectures https://review.openstack.org/582936 | 12:22 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/loci master: requirements.sh: parallel build only on x86-64 please https://review.openstack.org/582937 | 12:22 |
| hrw | SamYaple: if parallel is disabled then probably --no-deps can be removed as well | 12:24 |
| hrw | but I prefer to do small steps | 12:24 |
| *** ispp has joined #openstack-loci | 12:32 | |
| *** ispp has quit IRC | 13:15 | |
| *** ispp has joined #openstack-loci | 13:21 | |
| *** e0ne has quit IRC | 13:25 | |
| *** ispp has quit IRC | 13:28 | |
| *** ispp has joined #openstack-loci | 13:30 | |
| *** e0ne has joined #openstack-loci | 13:32 | |
| *** openstackgerrit has quit IRC | 14:04 | |
| *** ispp has quit IRC | 14:36 | |
| *** ispp has joined #openstack-loci | 14:40 | |
| *** ispp has quit IRC | 15:56 | |
| *** e0ne has quit IRC | 16:21 | |
| hrw | Successfully tagged requirements:master-ubuntu | 16:34 |
| hrw | on arm64 | 16:34 |
| hrw | had to kill parallel thing | 16:34 |
| hrw | hm. wrong | 16:35 |
| *** e0ne has joined #openstack-loci | 18:14 | |
| SamYaple | hrw: gema hello! | 18:43 |
| SamYaple | let me catch up | 18:43 |
| SamYaple | hrw: we dont pull from pypi... what does pypi only having x86 binaries matter? | 18:46 |
| SamYaple | the parallel thing for cassandra I was told to do that by someone who works on the project, but its a "requirement" :) | 18:47 |
| SamYaple | gema: hrw: based on your comments, im not sure you fully understand how the build works for LOCI | 18:49 |
| SamYaple | the changes you are talking about are not appropriate (nor issues currently), but im happy to answer design questions | 18:50 |
| SamYaple | perhaps we are missing something that arm needs... but i did cross compile at one time with no major issues (i think i needed to pull in the linaro repos for packages though) | 18:50 |
| *** e0ne has quit IRC | 19:05 | |
| gema | SamYaple: thanks for your reply | 19:17 |
| gema | SamYaple: we probably don't understand much yet, but the idea is to have the LOCI images buildable for Arm64 | 19:18 |
| gema | SamYaple: any comments you have or advice welcome, I will show up in tomorrow's weekly meeting | 19:18 |
| gema | and we can probably talk in a more synchronised way | 19:19 |
| gema | SamYaple: regarding your questions about the changes, I will let hrw take them, he is our official builder :D | 19:19 |
| SamYaple | cool. yes. im not being difficult, i think you may just be approaching it wrong | 19:21 |
| SamYaple | we definetely want teh same end goals and *will* get there | 19:21 |
| gema | SamYaple: sure, just be ruthless, no problems with that \o/ | 19:21 |
| gema | SamYaple: just put us in the right path | 19:21 |
| gema | :D | 19:21 |
| SamYaple | so the basic idea is all *building*, anything requiring dev libs, etc etc --- that all happens in the requirements image | 19:21 |
| SamYaple | regardless of achitecture | 19:21 |
| gema | SamYaple: so that is probably the one we need to make sure builds properly | 19:22 |
| SamYaple | only thing coming from pypi are 'whl' format wheels, which are architecture independant | 19:22 |
| SamYaple | correct | 19:22 |
| gema | ack | 19:22 |
| SamYaple | if requirements builds, the architecture is probably good | 19:22 |
| gema | SamYaple: and you don't have things like libvirt on any image? | 19:22 |
| SamYaple | gema: that is correct, i dont see what it would be needed | 19:23 |
| SamYaple | that said, https://github.com/openstack/loci/blob/master/bindep.txt thats the list of packages | 19:24 |
| SamYaple | (and bindep has architecture support) | 19:24 |
| SamYaple | so add/remove/rename as needed, specific for arm | 19:24 |
| gema | ack, that seems simple enough | 19:25 |
| gema | hrw: ^ for tomorrow x) | 19:25 |
| *** e0ne has joined #openstack-loci | 19:25 | |
| *** e0ne has quit IRC | 19:26 | |
| hrw | SamYaple: I have problems building loci/requirements:master-ubuntu image | 20:11 |
| hrw | SamYaple: and this image fetches from pypi | 20:12 |
| SamYaple | right, what issues are you having? | 20:15 |
| hrw | SamYaple: https://review.openstack.org/582935 adds libjpeg-dev(el) to build Pillow | 20:16 |
| SamYaple | and, while it does fetch from pypi, it doesnt fetch *binaries* from pypi, just native whl or source | 20:16 |
| hrw | SamYaple: https://review.openstack.org/582936 allows me to pass cryptography | 20:16 |
| hrw | SamYaple: scikit-term needs numpy and fails as it gets installed before numpy gets built (it takes quite a long time) | 20:17 |
| SamYaple | yes, but pip_install.sh is not meant to build anything, ever. thats what the requirements image is for | 20:17 |
| SamYaple | and since cryptography is in upper-constraints.txt, it should be in the requirements wheels archive for pip_install.sh (so no build tools needed for install) | 20:18 |
| hrw | SamYaple: and this is during build of requirements image | 20:18 |
| SamYaple | what do you mean? | 20:19 |
| hrw | docker build . --build-arg PROJECT=requirements --tag requirements:master-ubuntu 2>&1 |tee build-clean.log | 20:19 |
| hrw | will share a log once it finish | 20:20 |
| SamYaple | cool. that will be much more helpful | 20:20 |
| SamYaple | also, we are in no way tied to the current "logging" output from anything | 20:20 |
| SamYaple | we know how to read it, so we just sort of deal with it, but its admittedly not great | 20:20 |
| hrw | logs often suxx but better to have than not to | 20:21 |
| SamYaple | ya thats what we said too :) | 20:21 |
| hrw | ImportError: Numerical Python (NumPy) is not installed. | 20:33 |
| hrw | ImportError: Numerical Python (NumPy) is not installed. | 20:33 |
| hrw | scikit-learn requires NumPy >= 1.8.2. | 20:33 |
| hrw | that's one | 20:34 |
| hrw | at same time: | 20:34 |
| hrw | Running setup.py bdist_wheel for grpcio: still running... | 20:34 |
| hrw | Running setup.py bdist_wheel for lxml: still running... | 20:34 |
| hrw | Running setup.py bdist_wheel for PyNaCl: still running... | 20:34 |
| hrw | Running setup.py bdist_wheel for numpy: still running... | 20:34 |
| SamYaple | any chance to get the full log? | 20:34 |
| hrw | sure, once it end | 20:35 |
| SamYaple | cool | 20:35 |
| SamYaple | at the expense of a *lot* of time, you can change `-P20` to `-P1` to single thread it. https://github.com/openstack/loci/blob/master/scripts/requirements.sh#L47 | 20:36 |
| SamYaple | helps with the logs, but forces an hour long build | 20:36 |
| hrw | thanks! | 20:37 |
| hrw | will run such for a night | 20:37 |
| hrw | SamYaple: http://people.linaro.org/~marcin.juszkiewicz/loci/build-clean.log | 20:43 |
| hrw | shit. wrong one | 20:44 |
| hrw | http://people.linaro.org/~marcin.juszkiewicz/loci/build-origin-master.log is correct | 20:44 |
| hrw | started -P1 one | 20:44 |
| SamYaple | pyzmq issue is "zmq.h: No such file or directory" | 20:46 |
| SamYaple | youre welcome to explictly exclude broken libs if you dont have the deps for it | 20:47 |
| hrw | or add bindep | 20:47 |
| hrw | pyzmq should be buildable | 20:48 |
| hrw | SamYaple: btw - what time is at your place? | 20:50 |
| SamYaple | its not late, 1700 | 20:53 |
| hrw | US West like | 20:53 |
| hrw | 22:53 here. time to go to sleep soon | 20:54 |
| hrw | US East I mean ;d | 20:54 |
| SamYaple | haha yes, im US East, but i work US West hours remotely for a company | 20:58 |
| SamYaple | so hrw, i see the numpy issue | 20:58 |
| SamYaple | its not actually due to the parallism | 20:58 |
| SamYaple | and the issue can be fixed :) | 20:58 |
| SamYaple | the issue is that numpy is not available in the build env, which is seperate issue from *building* numpy, which we do in the requirements image | 20:59 |
| hrw | SamYaple: cool ;) | 20:59 |
| hrw | SamYaple: it is always better when someone knows what is going on and why | 20:59 |
| SamYaple | the version of numpy needed for building is unrelated, and the way it gets installed doesnt need version control (system packages in the base images of the x86 systems provide numpy, im guessing your base image doesnt) | 21:00 |
| SamYaple | either way, fixable. and easily :D | 21:00 |
| SamYaple | hrw: what the requirements image does is build *all* of upper-constraint, specific versions. It will compile those that do not have whls. There is also additional packages that can be added with build-args for packages not in upper-constraint | 21:02 |
| SamYaple | once they are built, the image essentially deletes everything but the built-wheels and becomes a tarball | 21:02 |
| SamYaple | that tarball we extract in the, for example, keystone image has all the wheels from upper-constraints already built. so we never install dev libs or build-tools in the keystone image | 21:03 |
| SamYaple | we dont actually install any of the wheels we build in the requirements image, we just build them | 21:04 |
| hrw | cool | 21:05 |
| hrw | thanks for simplifing description how it works ;) | 21:05 |
| hrw | SamYaple: if I could just do 'apt install python-numpy' etc I would be happy man ;D | 21:06 |
| SamYaple | https://review.openstack.org/#/c/506824/ | 21:08 |
| SamYaple | im just saying | 21:08 |
| SamYaple | its not a hard change for binary, i just dont have anyone who uses binary to drive it | 21:08 |
| SamYaple | and by drive, i really just mean test i guess | 21:09 |
| hrw | SamYaple: my final target will be Debian but as ubuntu is your default I want to get it working first | 21:10 |
| SamYaple | i use debian pretty heavily too. its just broken a few times recently | 21:11 |
| SamYaple | but its up-to-snuff | 21:11 |
| hrw | cool | 21:13 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!