Monday, 2017-06-19

ianwhmm, fdisk does an fsync on the file and a sync()00:00
ianwsync(3)                                = 000:00
ianwclose(3)                                = 000:01
ianwwrite(1, "Syncing disks.\n", 15)        = 1500:01
ianwsync()                                  = 000:01
ianwi think we'd do well to replicate that00:01
*** andreas-f has quit IRC00:32
*** openstackgerrit has joined #openstack-dib00:37
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Sync after writing partition table  https://review.openstack.org/47520300:37
ianwrm_work / johnsom: ^ with the benefit of hindsight, this does seem like a more likely cause ...00:37
johnsomThe other thought I had was to add a small retry loop here.00:40
ianwyeah, that seems a bit non-deterministic like a sleep() ... i'm more convinced about the sync() but still odd afaik we've never seen it once in any of our functional or nodepool tests, nor the infra builders hit it00:46
ianwoh dear ... debian has released00:48
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Install systemd earlier for strech  https://review.openstack.org/47520601:41
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Install systemd earlier for Stretch  https://review.openstack.org/47520601:42
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Install systemd earlier for Stretch  https://review.openstack.org/47520603:27
ianwok, next problem for today, nb03 is full04:48
ianw$ mount | wc -l04:49
ianw139004:49
ianwoh dear ... it's a mess04:49
ianwi'm rebooting it ... it's too far gone with 50 loop devices04:52
ianwok, it's building again with free space05:02
rm_work:)05:05
rm_workthanks for continuing to look at this issue05:05
rm_workthat seems sane, just need the gate issue to resolve and then recheck i guess ;P05:07
ianwrm_work: the gate should be fixed.  i tried using a depends-on but it didn't seem to work with the octavia job?05:15
ianwi couldn't see the sync call being made05:16
ianwthe global gate should be working i mean05:16
ianwdib is going to require the debian fix https://review.openstack.org/#/c/475206/05:16
ianwnot sure if anyone has an opinion on that05:16
*** andreas-f has joined #openstack-dib05:18
*** andreas-f has left #openstack-dib06:00
*** andreas-f has joined #openstack-dib06:00
ianwandreas-f: hmm, i'm not sure it's that different.  it pulls in sysv-init via package-installs anyway06:19
rm_workyeah i meant for https://review.openstack.org/#/c/475203/ to merge, it's going to need the gate issue resolved and then a recheck ;P06:22
ianwrm_work: yeah, the nodepool builder hit that problem too06:42
*** aparnav has joined #openstack-dib07:09
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Sync after writing partition table  https://review.openstack.org/47520307:13
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Sync after writing partition table  https://review.openstack.org/47520307:14
ianwyolanda: ^07:14
yolandaianw, just one change there?07:15
ianwoh, sorry, i just rebased that on 475206 which hasn't changed07:16
ianwjust so they can go into the gate in order07:16
yolandaok looking07:16
yolandahow is 475203 related to 475206?07:23
ianwit's not, just that it needs to go in to un break the gate07:31
*** openstackgerrit has quit IRC07:48
*** andreas-f has quit IRC09:31
*** owalsh_ is now known as owalsh10:06
*** hwoarang has quit IRC10:28
*** hwoarang has joined #openstack-dib10:29
*** amarao has joined #openstack-dib11:26
amaraoHello. I'm trying to migrate from dib 1.x to dib 2.6.0 and I found that after repackaging it as a deb-file (with same scripts) diskimage-create no longer works: disk-image-create11:28
amaraobash: /usr/lib/python2.7/dist-packages/diskimage_builder/lib/disk-image-create: No such file or directory. Where this file should come from?11:28
ianwamarao: the file layout has changed somewhat, deb packages would need to be updated11:49
amaraoYes, and I'm trying to understand what need to be changed.11:50
ianwhowever, it should all install correctly from setup.py with entry points, so really the package doesn't need to do much11:50
ianwamarao: see some of the comments in http://lists.openstack.org/pipermail/openstack-dev/2017-February/111833.html11:51
*** chhavi has joined #openstack-dib11:52
amaraook, thanks, I'd try to sort out it. Virtualenv as 'must do' is not look promising though...11:53
ianwamarao: it should actually be a lot easier.  "disk-image-create" should be in lib/ relative to where "import diskimage_builder" is ... you can see we just exec it in http://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/disk_image_create.py#n7211:57
ianwergo, "setup.py install" just works11:57
ianwi would reccommend a virtualenv approach, really, but some people don't ... infra install directly for example on nodepool builders11:58
amaraoWe are very oldschool and software should be packaged, and packaging should be scripted in CI server.11:58
amaraoapt-get install python-diskimage-builder.11:58
*** rfolco has joined #openstack-dib11:58
ianwthat's fine ... dh_python should "just work" ... nothing fancy going on11:59
amaraoUnfortunately it doesn't. That why I'm trying to figure out what happens.12:00
amarao%:12:00
amarao    dh $@ --buildsystem=python_distutils --with python12:00
amaraoThat's the main part. Plus few quirks to remove .eggs from source tree (after clean) and removal of *.pyc files. That's all.12:01
ianwonly thing i can think is that diskimage_builder/lib doesn't have an __init__.py12:03
ianwhave you got a package i can see?12:03
ianwamarao: oh, i assume you're using something based on  https://git.openstack.org/cgit/openstack/deb-python-diskimage-builder/tree/debian/rules?h=debian/newton12:14
*** farosas has joined #openstack-dib12:15
ianwi don't have time right now, but i can take a look later, i'd think using pybuild as a first step, it can also build 2&3 packages (python3 pretty well tested) and i wouldn't be surprised if distutils gets something wrong12:15
*** openstackgerrit has joined #openstack-dib13:22
openstackgerritMerged openstack/diskimage-builder master: Install systemd earlier for Stretch  https://review.openstack.org/47520613:22
*** aparnav has quit IRC13:24
*** chhavi has quit IRC16:36
openstackgerritMerged openstack/diskimage-builder master: Sync after writing partition table  https://review.openstack.org/47520316:49
*** amarao has quit IRC17:22
*** zigo has quit IRC18:05
*** zigo has joined #openstack-dib18:09
prometheanfireand amarao is gone :|18:12
*** jamielennox has quit IRC18:19
*** lennyb has quit IRC18:19
*** fungi has quit IRC18:19
*** jamielennox has joined #openstack-dib18:21
*** fungi has joined #openstack-dib18:25
*** lennyb has joined #openstack-dib18:27
*** dirk has joined #openstack-dib20:33
dirkHi20:33
openstackgerritMikhail S Medvedev proposed openstack/diskimage-builder master: DNM, Revert "Sync after writing partition table"  https://review.openstack.org/47552420:34
*** farosas has quit IRC21:02
ianwmmedvede: is it the stretch release?21:06
ianwrm_work / johnsom : ^^ things looking better with the sync calls in?21:06
johnsomToo early to tell I think.  We haven't been running a lot of jobs yet today, but some have gone in recently21:08
ianwok, fingers crossed21:08
ianwbecause i'm pretty low on other ideas of what it is ATM :)21:09
mmedvedeianw: not sure what is causing the failure I am seeing. Wasn't the latest patch. It looks like there is no /etc/modprobe.d dir now when the 15-cleanup-debootstrap is called21:14
mmedvedeit being the stretch release is a good guess21:15
*** rfolco has quit IRC21:21
*** mgagne has quit IRC21:21
*** mgagne has joined #openstack-dib21:22
*** mmedvede has left #openstack-dib21:30
*** mmedvede has joined #openstack-dib21:30

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!