| *** felipemonteiro__ has quit IRC | 00:01 | |
| *** mwarad has quit IRC | 00:01 | |
| *** baoli has quit IRC | 00:02 | |
| mordred | pabelanger: okie | 00:03 |
|---|---|---|
| *** dingyichen has joined #openstack-infra | 00:05 | |
| *** vhosakot_ has quit IRC | 00:06 | |
| *** rama_y has quit IRC | 00:07 | |
| *** baoli has joined #openstack-infra | 00:10 | |
| *** mriedem has quit IRC | 00:11 | |
| *** rhallisey has quit IRC | 00:12 | |
| *** bcat has quit IRC | 00:12 | |
| *** annegentle has quit IRC | 00:12 | |
| *** yamamoto_ has joined #openstack-infra | 00:13 | |
| *** yamamoto_ has quit IRC | 00:13 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix job timeout docs https://review.openstack.org/498513 | 00:14 |
| *** Swami has quit IRC | 00:19 | |
| *** ijw has joined #openstack-infra | 00:19 | |
| *** ijw has quit IRC | 00:21 | |
| *** signed8bit is now known as signed8bit_Zzz | 00:24 | |
| *** Julien-zte has quit IRC | 00:32 | |
| *** signed8bit_Zzz is now known as signed8bit | 00:34 | |
| *** Julien-zte has joined #openstack-infra | 00:34 | |
| *** jascott1 has quit IRC | 00:35 | |
| *** abelur_ has joined #openstack-infra | 00:37 | |
| *** jascott1 has joined #openstack-infra | 00:42 | |
| *** slaweq_ has joined #openstack-infra | 00:45 | |
| *** LindaWang has joined #openstack-infra | 00:45 | |
| *** slaweq_ has quit IRC | 00:50 | |
| *** signed8bit is now known as signed8bit_Zzz | 00:50 | |
| *** thorst_afk has joined #openstack-infra | 00:51 | |
| *** jamesmcarthur has joined #openstack-infra | 00:52 | |
| *** zhurong has joined #openstack-infra | 00:52 | |
| *** jamesmcarthur has quit IRC | 00:56 | |
| *** liujiong has joined #openstack-infra | 00:56 | |
| *** mat128 has joined #openstack-infra | 00:57 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add role for adding launchpadlib credentials https://review.openstack.org/498633 | 00:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add tag-release job https://review.openstack.org/498634 | 00:58 |
| kong | hi, guys, I retried to create devstack-gate image, but still failed to start a vm. | 00:58 |
| mordred | jeblair, pabelanger: ^^ there's two patches that should get us tag-release | 00:59 |
| kong | the console of the vm says: no bootable device, https://pasteboard.co/GHORwNI.png | 00:59 |
| kong | This is the last portion of build-image.sh log: http://dpaste.com/3FTWC6B | 00:59 |
| kong | hope some could help me | 00:59 |
| *** thorst_afk has quit IRC | 00:59 | |
| clarkb | kong: are tou able to boot that locally using kvm? | 01:00 |
| clarkb | I would start there | 01:00 |
| kong | clarkb: didn't try yet. do now | 01:01 |
| *** baoli has quit IRC | 01:01 | |
| *** masber has quit IRC | 01:01 | |
| *** baoli has joined #openstack-infra | 01:02 | |
| *** masber has joined #openstack-infra | 01:02 | |
| clarkb | kong: the other thing is what elements list are you using? these are passed as args to dib on invocation | 01:04 |
| kong | clarkb: i just called 'tools/build-image.sh' with no args | 01:05 |
| *** gouthamr_ has joined #openstack-infra | 01:05 | |
| clarkb | in project-config? | 01:05 |
| kong | yes | 01:05 |
| *** baoli has quit IRC | 01:06 | |
| kong | clarkb: according to the doc: https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/README.rst#n111 | 01:06 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Write a logging config file and pass it to callbacks https://review.openstack.org/498127 | 01:06 |
| *** gouthamr has quit IRC | 01:07 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Remove default root handler fallback to console https://review.openstack.org/498635 | 01:07 |
| *** skelso has joined #openstack-infra | 01:10 | |
| *** sekelso has joined #openstack-infra | 01:12 | |
| *** yamamoto_ has joined #openstack-infra | 01:14 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Add release-announce job https://review.openstack.org/498623 | 01:14 |
| *** aviau has quit IRC | 01:15 | |
| *** skelso has quit IRC | 01:15 | |
| *** aviau has joined #openstack-infra | 01:15 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add role for adding launchpadlib credentials https://review.openstack.org/498633 | 01:16 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add trigger-readthedocs job https://review.openstack.org/498626 | 01:17 |
| *** baoli has joined #openstack-infra | 01:18 | |
| *** signed8bit_Zzz is now known as signed8bit | 01:19 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Add release-announce job https://review.openstack.org/498623 | 01:20 |
| *** yamamoto_ has quit IRC | 01:20 | |
| clarkb | kong: also check the md5sum for the local package and in glance | 01:20 |
| kong | clarkb: in terms of my steps, if there is something i did wrong? | 01:21 |
| clarkb | no running the image build script should produce a bootable image | 01:22 |
| kong | the checksum is correct | 01:23 |
| clarkb | but trying to track down where it could be failing | 01:23 |
| kong | clarkb: i will try to see how to boot a vm using kvm directly | 01:24 |
| clarkb | kong: ok, I use virt-install myself | 01:24 |
| clarkb | which I think reauires virsh | 01:24 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/devstack-gate master: Move fix_disk_layout to an Ansible role in setup_host https://review.openstack.org/496935 | 01:25 |
| *** gouthamr_ is now known as gouthamr | 01:27 | |
| *** camunoz has quit IRC | 01:36 | |
| clarkb | kong: another thing you might check is that your flavor has enough disk for the image | 01:37 |
| clarkb | I'm not sure what happens when it is too small but wouldnt surprise me if this is the behavior | 01:37 |
| kong | clarkb: i am using 2 cpu, 8G ram and 10G disk | 01:39 |
| *** cuongnv has joined #openstack-infra | 01:41 | |
| clarkb | 10GB might not be enough. th image is close to that size iirc and represents a larger disk | 01:41 |
| clarkb | kong: I would try 20GB if you are able just to test | 01:42 |
| kong | clarkb: really? | 01:43 |
| kong | clarkb: the image info: http://dpaste.com/1R4M5SG | 01:44 |
| ianw | kong: yeah try all that, also "50-bootloader 72.208" ... that seems like a lot of seconds for that to run. if you can gz the full log we might see something | 01:44 |
| clarkb | that image seems small but maybe it didnt cache the git repos | 01:44 |
| ianw | clarkb: i think kong was also using the clone list stripping thing in that tools/build-image to strip out most repos | 01:44 |
| clarkb | (in which case it should boot fine) | 01:44 |
| kong | ianw, clarkb, the only change i made is to enable export DIB_CUSTOM_PROJECTS_LIST_URL='file:///tmp/custom_projects.yaml' | 01:45 |
| *** slaweq_ has joined #openstack-infra | 01:46 | |
| ianw | yeah, that will just stop cloning all the git repos (which may not be what you want exactly, but anyway shouldn't affect bootup) | 01:46 |
| ianw | kong: you can also strip this right down, and just do a "disk-image-create centos-minimal vm" and see if that boots (it won't have networking, or users, but can worry about that later) | 01:46 |
| kong | i enabled that just to avoid hitting the cirros-cloud issue | 01:47 |
| *** ijw has joined #openstack-infra | 01:48 | |
| ianw | or ubuntu-minimal, i guess from that log you're building xenial? | 01:50 |
| *** slaweq_ has quit IRC | 01:50 | |
| *** spzala has quit IRC | 01:50 | |
| kong | ianw: yeah, seems ubuntu-minimal is the default value in build-image.sh | 01:52 |
| *** jascott1 has quit IRC | 01:53 | |
| *** ijw has quit IRC | 01:53 | |
| ianw | kong: and you're building on a xenial host system? | 01:53 |
| *** jascott1 has joined #openstack-infra | 01:53 | |
| kong | ianw: no, trusty | 01:54 |
| ianw | ok, well that should work too, dib still tests itself on trusty (though i'm thinking of removing it, as nodepool builders are now all xenial) | 01:55 |
| *** jascott1 has quit IRC | 01:57 | |
| *** thorst_afk has joined #openstack-infra | 02:00 | |
| *** thorst_afk has quit IRC | 02:06 | |
| *** iyamahat has quit IRC | 02:08 | |
| *** yamahata has quit IRC | 02:08 | |
| *** signed8b_ has joined #openstack-infra | 02:14 | |
| *** baoli has quit IRC | 02:16 | |
| *** yamamoto_ has joined #openstack-infra | 02:16 | |
| *** signed8bit has quit IRC | 02:17 | |
| *** baoli has joined #openstack-infra | 02:19 | |
| *** baoli has quit IRC | 02:19 | |
| *** baoli has joined #openstack-infra | 02:19 | |
| *** ijw has joined #openstack-infra | 02:20 | |
| *** yamamoto_ has quit IRC | 02:22 | |
| *** signed8b_ has quit IRC | 02:23 | |
| *** signed8bit has joined #openstack-infra | 02:24 | |
| *** ijw has quit IRC | 02:28 | |
| *** Julien-zte has quit IRC | 02:31 | |
| *** dave-mccowan has quit IRC | 02:32 | |
| *** jascott1 has joined #openstack-infra | 02:33 | |
| *** xinliang has quit IRC | 02:36 | |
| *** jascott1 has quit IRC | 02:37 | |
| *** ramishra has joined #openstack-infra | 02:38 | |
| *** jbernard has quit IRC | 02:40 | |
| *** hongbin has joined #openstack-infra | 02:43 | |
| *** signed8bit is now known as signed8bit_Zzz | 02:44 | |
| *** sree has joined #openstack-infra | 02:45 | |
| *** slaweq_ has joined #openstack-infra | 02:47 | |
| *** xinliang has joined #openstack-infra | 02:49 | |
| *** xinliang has quit IRC | 02:49 | |
| *** xinliang has joined #openstack-infra | 02:49 | |
| *** sree has quit IRC | 02:49 | |
| *** yamamoto has joined #openstack-infra | 02:49 | |
| *** gcb has joined #openstack-infra | 02:50 | |
| *** felipemonteiro__ has joined #openstack-infra | 02:51 | |
| *** slaweq_ has quit IRC | 02:52 | |
| *** masber has quit IRC | 02:53 | |
| *** rlandy|bbl is now known as rlandy | 02:54 | |
| *** felipemonteiro_ has joined #openstack-infra | 02:55 | |
| *** hongbin_ has joined #openstack-infra | 02:56 | |
| *** hongbin has quit IRC | 02:56 | |
| *** hongbin_ has quit IRC | 02:56 | |
| *** hongbin has joined #openstack-infra | 02:56 | |
| *** felipemonteiro has joined #openstack-infra | 02:57 | |
| *** felipemonteiro__ has quit IRC | 02:59 | |
| *** jamesmcarthur has joined #openstack-infra | 02:59 | |
| *** gouthamr has quit IRC | 02:59 | |
| *** felipemonteiro_ has quit IRC | 03:00 | |
| *** thorst_afk has joined #openstack-infra | 03:01 | |
| kong | hi, ianw, clarkb, sorry to bother you again, but it still failed with 'no bootable device' error even i was running 'disk-image-create ubuntu-minimal vm' | 03:03 |
| kong | log here: http://dpaste.com/0GY9J2A | 03:04 |
| kong | this time, i didn't change anything in project-config repo | 03:05 |
| *** thorst_afk has quit IRC | 03:06 | |
| *** sekelso has quit IRC | 03:06 | |
| *** yamahata has joined #openstack-infra | 03:06 | |
| *** signed8bit_Zzz is now known as signed8bit | 03:10 | |
| *** felipemonteiro has quit IRC | 03:11 | |
| *** yamamoto has quit IRC | 03:12 | |
| clarkb | kong: ok that tells us it isnt anything to do with the infra stuff | 03:18 |
| clarkb | since that is base disk image builder | 03:19 |
| clarkb | kong maybe try mount the qcow2 image and see if it looks funny? | 03:19 |
| clarkb | ianw probably knows where to look in the log better than me | 03:19 |
| *** signed8b_ has joined #openstack-infra | 03:24 | |
| kong | ok, thanks clarkb | 03:27 |
| *** signed8bit has quit IRC | 03:27 | |
| *** jbernard has joined #openstack-infra | 03:27 | |
| *** ijw has joined #openstack-infra | 03:29 | |
| *** baoli has quit IRC | 03:30 | |
| *** annegentle has joined #openstack-infra | 03:34 | |
| *** jamesmcarthur has quit IRC | 03:39 | |
| *** ijw has quit IRC | 03:40 | |
| *** ijw has joined #openstack-infra | 03:40 | |
| *** ijw_ has joined #openstack-infra | 03:41 | |
| ianw | kong: ok, juts give me a few and i'll take a look... | 03:43 |
| *** jascott1 has joined #openstack-infra | 03:44 | |
| *** ijw has quit IRC | 03:44 | |
| *** slaweq_ has joined #openstack-infra | 03:48 | |
| *** annegentle has quit IRC | 03:48 | |
| *** annegentle has joined #openstack-infra | 03:48 | |
| *** udesale has joined #openstack-infra | 03:50 | |
| *** sree has joined #openstack-infra | 03:50 | |
| kong | ianw: appricate! | 03:50 |
| *** slaweq_ has quit IRC | 03:52 | |
| *** annegentle has quit IRC | 03:53 | |
| *** jascott1 has quit IRC | 03:53 | |
| ianw | /tmp/in_target.d/finalise.d/50-bootloader: line 160: type: grub2-mkconfig: not found | 03:55 |
| ianw | hmm, suspicious | 03:55 |
| ianw | probably ok ... | 03:58 |
| ianw | kong: what do you do then? upload to glance? | 03:58 |
| kong | ianw: yes, upload, create vm | 03:59 |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Support Zuul v3 repo layout https://review.openstack.org/497618 | 04:00 |
| ianw | kong: and if you just get a standard upstream image, that boots, right? | 04:01 |
| kong | ianw: yes | 04:02 |
| *** thorst_afk has joined #openstack-infra | 04:02 | |
| kong | i just want to get a devstack-gate image and test my project gate script | 04:02 |
| ianw | kong: ok, can you put that minimal .qcow somewhere for download? | 04:02 |
| *** links has joined #openstack-infra | 04:03 | |
| kong | ianw: sure, wait a sec | 04:03 |
| ianw | kong: sure, i mean this should build fine ... i use it quite a bit. so interested in making sure we've caught corner cases | 04:03 |
| *** thorst_afk has quit IRC | 04:07 | |
| *** masber has joined #openstack-infra | 04:08 | |
| *** adisky__ has joined #openstack-infra | 04:11 | |
| *** yamamoto has joined #openstack-infra | 04:13 | |
| *** hongbin has quit IRC | 04:13 | |
| kong | ianw: please see pm | 04:14 |
| *** jascott1 has joined #openstack-infra | 04:18 | |
| *** yamamoto has quit IRC | 04:19 | |
| *** shu-mutou-AWAY is now known as shu-mutou | 04:20 | |
| *** claudiub|2 has joined #openstack-infra | 04:21 | |
| *** zhurong has quit IRC | 04:21 | |
| *** ykarel has joined #openstack-infra | 04:23 | |
| *** mwarad has joined #openstack-infra | 04:30 | |
| *** arturb has joined #openstack-infra | 04:33 | |
| ianw | kong: ok, i think it's something local to your environment stopping it booting. the image you gave me boots to a login prompt | 04:37 |
| ianw | i uploaded it with "image create "testing" --file ./image.qcow2 --disk-format=qcow2 --container-format bare" | 04:37 |
| *** felipemonteiro has joined #openstack-infra | 04:37 | |
| *** calebb has quit IRC | 04:44 | |
| *** rama_y has joined #openstack-infra | 04:45 | |
| kong | s**t, i didn't specify the format | 04:47 |
| kong | ianw: thanks so much! | 04:47 |
| *** felipemonteiro has quit IRC | 04:47 | |
| kong | i am sure we should fix: https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/README.rst#n168 | 04:48 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: the kolla container playbook needs release variables https://review.openstack.org/498688 | 04:48 |
| *** slaweq_ has joined #openstack-infra | 04:49 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Add upload instructions to build-image.sh https://review.openstack.org/498689 | 04:53 |
| ianw | kong: probably; i also added ^ | 04:54 |
| *** mpeterson has joined #openstack-infra | 04:54 | |
| *** slaweq_ has quit IRC | 04:54 | |
| *** calebb has joined #openstack-infra | 04:54 | |
| kong | ianw: thanks for saving my life :-) | 04:54 |
| *** zhurong has joined #openstack-infra | 04:56 | |
| *** SotK has quit IRC | 04:58 | |
| *** SotK has joined #openstack-infra | 04:58 | |
| ianw | i'm not sure i'd go that far ;) | 05:01 |
| *** thorst_afk has joined #openstack-infra | 05:03 | |
| *** kornicameister has quit IRC | 05:03 | |
| *** thorst_afk has quit IRC | 05:08 | |
| *** pbourke has quit IRC | 05:10 | |
| *** pbourke has joined #openstack-infra | 05:12 | |
| *** signed8b_ has quit IRC | 05:14 | |
| *** yamamoto has joined #openstack-infra | 05:15 | |
| *** cuongnv has quit IRC | 05:20 | |
| *** yamamoto has quit IRC | 05:21 | |
| *** cuongnv has joined #openstack-infra | 05:24 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove V1 driver gate jobs https://review.openstack.org/497696 | 05:26 |
| *** ijw_ has quit IRC | 05:31 | |
| *** pgadiya has joined #openstack-infra | 05:36 | |
| AJaeger | ianw, could you review a cosmetic infra-manual change as time permits, please? https://review.openstack.org/497532 | 05:39 |
| *** cuongnv has quit IRC | 05:41 | |
| *** cuongnv has joined #openstack-infra | 05:42 | |
| *** ykarel_ has joined #openstack-infra | 05:49 | |
| *** slaweq_ has joined #openstack-infra | 05:50 | |
| *** ykarel has quit IRC | 05:51 | |
| openstackgerrit | Merged openstack-infra/project-config master: kuryr octavia job: add barbican to PROJECTS https://review.openstack.org/498406 | 05:51 |
| ianw | heh the ` annoys me too :) | 05:54 |
| *** slaweq_ has quit IRC | 05:54 | |
| AJaeger | ianw: thanks. I didn't notice them the first round and then had to clean up myself ;) | 05:57 |
| openstackgerrit | Merged openstack-infra/infra-manual master: Fix some markup in v3 docu https://review.openstack.org/497532 | 05:57 |
| openstackgerrit | Merged openstack-infra/project-config master: Add python 3.6 job for python-swiftclient https://review.openstack.org/497979 | 05:58 |
| openstackgerrit | Merged openstack/diskimage-builder master: Use [[ for =~ matches https://review.openstack.org/498333 | 05:59 |
| *** ykarel_ is now known as ykarel | 06:00 | |
| *** lihi has joined #openstack-infra | 06:00 | |
| *** thorst_afk has joined #openstack-infra | 06:03 | |
| *** eranrom has joined #openstack-infra | 06:05 | |
| *** thorst_afk has quit IRC | 06:09 | |
| *** yamamoto has joined #openstack-infra | 06:13 | |
| *** edisonxiang has quit IRC | 06:14 | |
| *** edisonxiang has joined #openstack-infra | 06:14 | |
| *** mat128 has quit IRC | 06:16 | |
| *** pgadiya has quit IRC | 06:18 | |
| *** rlandy has quit IRC | 06:27 | |
| *** martinkopec has joined #openstack-infra | 06:28 | |
| *** iyamahat has joined #openstack-infra | 06:29 | |
| *** iyamahat_ has joined #openstack-infra | 06:30 | |
| *** iyamahat has quit IRC | 06:30 | |
| *** iyamahat_ has quit IRC | 06:31 | |
| *** pgadiya has joined #openstack-infra | 06:34 | |
| *** yamamoto has quit IRC | 06:38 | |
| *** yamamoto has joined #openstack-infra | 06:39 | |
| *** yamamoto has quit IRC | 06:39 | |
| *** pcaruana has joined #openstack-infra | 06:39 | |
| *** mpeterson has quit IRC | 06:44 | |
| *** markus_z has joined #openstack-infra | 06:45 | |
| *** raissa has quit IRC | 06:46 | |
| *** slaweq_ has joined #openstack-infra | 06:51 | |
| *** coolsvap has quit IRC | 06:52 | |
| *** mpeterso1 has joined #openstack-infra | 06:52 | |
| *** kjackal has joined #openstack-infra | 06:53 | |
| *** pgadiya has quit IRC | 06:54 | |
| *** andreas_s has joined #openstack-infra | 06:54 | |
| *** slaweq_ has quit IRC | 06:56 | |
| *** dhajare has joined #openstack-infra | 06:57 | |
| *** eumel8_away is now known as eumel8 | 06:58 | |
| *** thorst_afk has joined #openstack-infra | 07:05 | |
| *** pgadiya has joined #openstack-infra | 07:06 | |
| *** rcernin has joined #openstack-infra | 07:10 | |
| *** thorst_afk has quit IRC | 07:10 | |
| *** florianf has joined #openstack-infra | 07:13 | |
| *** zhurong has quit IRC | 07:15 | |
| *** zhurong has joined #openstack-infra | 07:17 | |
| *** sshnaidm|afk is now known as sshnaidm | 07:19 | |
| *** jascott1 has quit IRC | 07:23 | |
| *** jascott1 has joined #openstack-infra | 07:24 | |
| *** tesseract has joined #openstack-infra | 07:25 | |
| *** Hal has joined #openstack-infra | 07:28 | |
| *** jpena|off is now known as jpena | 07:28 | |
| *** jascott1 has quit IRC | 07:28 | |
| *** Hal is now known as Guest54308 | 07:28 | |
| *** jpich has joined #openstack-infra | 07:32 | |
| *** ykarel is now known as ykarel|lunch | 07:33 | |
| *** hashar has joined #openstack-infra | 07:34 | |
| *** hashar has quit IRC | 07:34 | |
| *** hashar has joined #openstack-infra | 07:34 | |
| *** egonzalez has joined #openstack-infra | 07:36 | |
| *** jtomasek has quit IRC | 07:37 | |
| *** jtomasek_ has joined #openstack-infra | 07:37 | |
| *** yamahata has quit IRC | 07:37 | |
| *** rossella_s has joined #openstack-infra | 07:40 | |
| *** yamamoto has joined #openstack-infra | 07:40 | |
| *** priteau has joined #openstack-infra | 07:40 | |
| *** hashar has quit IRC | 07:41 | |
| *** hashar has joined #openstack-infra | 07:41 | |
| *** yamamoto has quit IRC | 07:45 | |
| *** ralonsoh has joined #openstack-infra | 07:50 | |
| *** slaweq_ has joined #openstack-infra | 07:52 | |
| *** slaweq_ has quit IRC | 07:57 | |
| *** thorst_afk has joined #openstack-infra | 08:05 | |
| *** alex_xu has quit IRC | 08:06 | |
| *** thorst_afk has quit IRC | 08:10 | |
| *** alex_xu has joined #openstack-infra | 08:12 | |
| *** alexchadin has joined #openstack-infra | 08:12 | |
| *** raissa has joined #openstack-infra | 08:20 | |
| *** bogdando has joined #openstack-infra | 08:29 | |
| sshnaidm | hi, is the gate long queue known issue? | 08:36 |
| *** efoley has joined #openstack-infra | 08:40 | |
| *** yamamoto has joined #openstack-infra | 08:42 | |
| *** dingyichen has quit IRC | 08:43 | |
| *** yamamoto has quit IRC | 08:47 | |
| *** bogdando has quit IRC | 08:48 | |
| *** slaweq_ has joined #openstack-infra | 08:53 | |
| *** bogdando has joined #openstack-infra | 08:53 | |
| *** tosky has joined #openstack-infra | 08:55 | |
| *** electrofelix has joined #openstack-infra | 08:55 | |
| openstackgerrit | Gao Zexu proposed openstack/os-testr master: Update the documentation urls https://review.openstack.org/498714 | 08:57 |
| *** ykarel|lunch is now known as ykarel | 08:57 | |
| *** slaweq_ has quit IRC | 08:58 | |
| *** lucas-afk is now known as lucasagomes | 08:59 | |
| *** thorst_afk has joined #openstack-infra | 09:06 | |
| *** thorst_afk has quit IRC | 09:11 | |
| bogdando | hi. Can we merge e-r queries https://review.openstack.org/#/c/493535/ https://review.openstack.org/#/c/493525/ to reduce unknown/uncategorised numbers? | 09:12 |
| bogdando | nevermind, it's not a relevant thing anymore | 09:17 |
| *** bogdando has left #openstack-infra | 09:17 | |
| *** sshnaidm is now known as sshnaidm|afk | 09:18 | |
| kklimonda | is there an example of a diskimage-builder workflow, with custom package lists, repositories, additional config and elements? I'm interested in how to keep all customizations in repository, so it can be reviewed etc. Is openstack/tripleo-image-elements a good place to start? | 09:23 |
| *** dhajare has quit IRC | 09:24 | |
| *** eranrom has quit IRC | 09:28 | |
| *** gixxer1k has joined #openstack-infra | 09:30 | |
| *** sambetts|afk is now known as sambetts | 09:36 | |
| *** dhajare has joined #openstack-infra | 09:36 | |
| *** mpeterso1 has quit IRC | 09:41 | |
| *** yamamoto has joined #openstack-infra | 09:43 | |
| *** mwarad has quit IRC | 09:45 | |
| *** yamamoto has quit IRC | 09:49 | |
| *** slaweq_ has joined #openstack-infra | 09:54 | |
| *** sdague has joined #openstack-infra | 09:54 | |
| *** mpeterso1 has joined #openstack-infra | 09:56 | |
| *** mpeterso1 has quit IRC | 09:57 | |
| *** alexchadin has quit IRC | 09:58 | |
| *** slaweq_ has quit IRC | 09:58 | |
| *** alexchadin has joined #openstack-infra | 09:59 | |
| *** cuongnv has quit IRC | 10:01 | |
| *** udesale has quit IRC | 10:02 | |
| *** rook is now known as rook|pto | 10:06 | |
| *** liujiong has quit IRC | 10:11 | |
| *** alexchadin has quit IRC | 10:12 | |
| *** alexchadin has joined #openstack-infra | 10:13 | |
| *** alexchadin has quit IRC | 10:13 | |
| *** alexchadin has joined #openstack-infra | 10:14 | |
| *** shu-mutou is now known as shu-mutou-AWAY | 10:17 | |
| *** alexchadin has quit IRC | 10:17 | |
| *** sshnaidm|afk is now known as sshnaidm | 10:19 | |
| *** dtantsur|afk is now known as dtantsur | 10:23 | |
| *** ociuhandu has quit IRC | 10:30 | |
| *** zhurong has quit IRC | 10:35 | |
| *** pbourke has quit IRC | 10:35 | |
| *** pbourke has joined #openstack-infra | 10:37 | |
| *** pbourke has quit IRC | 10:39 | |
| kklimonda | I see that there is https://github.com/openstack-infra/project-config/blob/master/tools/build-image.sh but it's suspiciously specific to ubuntu 14.04 release | 10:39 |
| *** kjackal has quit IRC | 10:40 | |
| *** stakeda has quit IRC | 10:41 | |
| *** felipemonteiro has joined #openstack-infra | 10:44 | |
| *** yamamoto has joined #openstack-infra | 10:45 | |
| *** jkilpatr has quit IRC | 10:46 | |
| *** ykarel is now known as ykarel|afk | 10:46 | |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci master: Use release specific repos during containers build https://review.openstack.org/498688 | 10:47 |
| kklimonda | ah, I just parsed it | 10:47 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Add gearman server port configuration https://review.openstack.org/498753 | 10:47 |
| *** felipemonteiro has quit IRC | 10:48 | |
| *** yamamoto has quit IRC | 10:51 | |
| *** ociuhandu has joined #openstack-infra | 10:51 | |
| *** e0ne has joined #openstack-infra | 10:52 | |
| *** wolverineav has joined #openstack-infra | 10:53 | |
| *** hashar is now known as hasharLucnh | 10:54 | |
| *** hasharLucnh is now known as hasharLunch | 10:54 | |
| *** slaweq_ has joined #openstack-infra | 10:55 | |
| *** e0ne has quit IRC | 10:56 | |
| *** e0ne has joined #openstack-infra | 10:57 | |
| *** slaweq_ has quit IRC | 11:00 | |
| *** jkilpatr has joined #openstack-infra | 11:04 | |
| *** thorst_afk has joined #openstack-infra | 11:07 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Add gearman server port configuration https://review.openstack.org/498753 | 11:08 |
| *** thorst_afk has quit IRC | 11:12 | |
| *** alexchadin has joined #openstack-infra | 11:13 | |
| *** e0ne has quit IRC | 11:14 | |
| *** alexchadin has quit IRC | 11:16 | |
| *** pbourke has joined #openstack-infra | 11:20 | |
| *** kjackal has joined #openstack-infra | 11:22 | |
| *** alexchadin has joined #openstack-infra | 11:24 | |
| *** e0ne has joined #openstack-infra | 11:26 | |
| *** martinkopec has quit IRC | 11:26 | |
| *** martinkopec has joined #openstack-infra | 11:28 | |
| *** sree has quit IRC | 11:28 | |
| *** kjackal has quit IRC | 11:28 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack-infra/elastic-recheck master: Add query for bug 1713167 https://review.openstack.org/498766 | 11:29 |
| openstack | bug 1713167 in tripleo "build-images : Throw error msg if image build is failing *** Image Build Error ***" [High,Triaged] https://launchpad.net/bugs/1713167 | 11:29 |
| openstackgerrit | Bogdan Dobrelya proposed openstack-infra/elastic-recheck master: Add query for bug 1713703 https://review.openstack.org/498766 | 11:31 |
| openstack | bug 1713703 in OpenStack-Gate "Generic intermittent connection issue to github.com" [Undecided,New] https://launchpad.net/bugs/1713703 | 11:31 |
| *** ldnunes has joined #openstack-infra | 11:32 | |
| *** zhurong has joined #openstack-infra | 11:33 | |
| *** hasharLunch is now known as hashar | 11:35 | |
| *** udesale has joined #openstack-infra | 11:36 | |
| *** pgadiya has quit IRC | 11:39 | |
| *** Guest54308 has quit IRC | 11:40 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Multinode jobs should not build images https://review.openstack.org/498772 | 11:40 |
| *** ykarel|afk is now known as ykarel | 11:46 | |
| *** rhallisey has joined #openstack-infra | 11:46 | |
| *** trown|outtypewww is now known as trown | 11:47 | |
| *** yamamoto has joined #openstack-infra | 11:47 | |
| *** yamamoto has quit IRC | 11:52 | |
| *** gixxer1k has quit IRC | 11:55 | |
| *** gixxer1k has joined #openstack-infra | 11:55 | |
| *** ldnunes has quit IRC | 11:57 | |
| *** lucasagomes is now known as lucas-hungry | 11:58 | |
| *** martinkopec has quit IRC | 12:00 | |
| *** martinkopec has joined #openstack-infra | 12:01 | |
| *** martinkopec has quit IRC | 12:01 | |
| *** martinkopec has joined #openstack-infra | 12:02 | |
| *** dave-mccowan has joined #openstack-infra | 12:06 | |
| *** thorst_afk has joined #openstack-infra | 12:06 | |
| *** ldnunes has joined #openstack-infra | 12:10 | |
| *** jpena is now known as jpena|lunch | 12:14 | |
| wendar | pabelanger: heads-up, I'm working on the patch to remove Debian Packaging project from gating, should have it in gerrit later today | 12:15 |
| wendar | pabelanger: (trying to minimize conflicts) | 12:16 |
| *** dtantsur is now known as dtantsur|busy | 12:23 | |
| *** martinkopec has quit IRC | 12:28 | |
| *** Hal has joined #openstack-infra | 12:29 | |
| *** Hal is now known as Guest84763 | 12:29 | |
| *** gcb has quit IRC | 12:29 | |
| *** gcb has joined #openstack-infra | 12:30 | |
| *** markus_z has quit IRC | 12:36 | |
| *** zhurong has quit IRC | 12:36 | |
| *** alexchadin has quit IRC | 12:36 | |
| *** ykarel has quit IRC | 12:36 | |
| *** martinkopec has joined #openstack-infra | 12:38 | |
| fungi | wendar: thanks for working on that! | 12:39 |
| *** rlandy has joined #openstack-infra | 12:41 | |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Add new repo for aodh tempest plugin https://review.openstack.org/498796 | 12:43 |
| *** LindaWang has quit IRC | 12:44 | |
| wendar | fungi: glad to help! (and glad to clean it up too) | 12:44 |
| *** felipemonteiro has joined #openstack-infra | 12:46 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci master: WIP -- switch multinode composable upgrade job to oooq https://review.openstack.org/463580 | 12:47 |
| *** mat128 has joined #openstack-infra | 12:47 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci master: WIP -- switch multinode composable upgrade job to oooq https://review.openstack.org/463580 | 12:47 |
| *** slaweq_ has joined #openstack-infra | 12:48 | |
| *** slaweq_ has quit IRC | 12:48 | |
| *** yamamoto has joined #openstack-infra | 12:49 | |
| *** felipemonteiro_ has joined #openstack-infra | 12:49 | |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Add new repo for aodh tempest plugin https://review.openstack.org/498796 | 12:49 |
| *** sree has joined #openstack-infra | 12:50 | |
| *** felipemonteiro__ has joined #openstack-infra | 12:50 | |
| *** eharney has quit IRC | 12:50 | |
| *** zhurong has joined #openstack-infra | 12:50 | |
| *** alexchadin has joined #openstack-infra | 12:50 | |
| *** kjackal has joined #openstack-infra | 12:52 | |
| *** felipemonteiro has quit IRC | 12:52 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Include the prepared projects list in zuul variables https://review.openstack.org/498618 | 12:53 |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config master: Add networking-cisco UTs against neutron stable/pike https://review.openstack.org/498801 | 12:53 |
| *** bh526r has joined #openstack-infra | 12:53 | |
| *** yamamoto has quit IRC | 12:54 | |
| *** bobh has joined #openstack-infra | 12:54 | |
| *** felipemonteiro_ has quit IRC | 12:54 | |
| *** lucas-hungry is now known as lucasagomes | 13:01 | |
| *** jcoufal has joined #openstack-infra | 13:01 | |
| *** slaweq has quit IRC | 13:02 | |
| *** esberglu has joined #openstack-infra | 13:02 | |
| *** skelso has joined #openstack-infra | 13:02 | |
| *** pblaho has quit IRC | 13:02 | |
| *** esberglu has quit IRC | 13:02 | |
| *** esberglu has joined #openstack-infra | 13:03 | |
| *** slaweq has joined #openstack-infra | 13:03 | |
| AJaeger | wendar: I'm happy to prioritize it. Do you want to do one repo first so that we can iteratate on a smaller scale? Once that change for a repo is good, we can look at the rest | 13:06 |
| wendar | AJaeger: I'm starting with ending gating | 13:07 |
| wendar | AJaeger: That's the one with the greatest risk of conflicts | 13:07 |
| wendar | AJaeger: after that goes through, yeah, I'll step through one at a time on the project repos | 13:07 |
| wendar | AJaeger: and, yeah, doing one first makes sense, to get the review on whether it's done right, before repeating hundreds of times :) | 13:07 |
| AJaeger | wendar: just ask here if you want reviews, when I have time, I'll review quickly | 13:08 |
| wendar | AJaeger: thanks! just about to put the first one into gerrit | 13:09 |
| chandankumar | AJaeger: please have a look at this review https://review.openstack.org/#/c/498796/ i am not getting why irc job is failing | 13:11 |
| chandankumar | please have a look. | 13:11 |
| *** krtaylor has joined #openstack-infra | 13:11 | |
| openstackgerrit | Allison Randal proposed openstack-infra/project-config master: Remove Packaging Deb project from gate https://review.openstack.org/498807 | 13:11 |
| wendar | AJaeger/pabelanger: and, it's in. Lots of lines but mostly just small repetitive changes/removals across a large number of repos in the project. | 13:12 |
| *** LindaWang has joined #openstack-infra | 13:13 | |
| AJaeger | wendar: let's see whether our jobs pass ;) | 13:13 |
| wendar | AJaeger: nod :) | 13:13 |
| *** kgiusti has joined #openstack-infra | 13:13 | |
| AJaeger | wendar: if they pass, let'S merge it - you get my vote of confidence ( +2 ) | 13:14 |
| wendar | AJaeger: thanks! :) | 13:15 |
| *** mriedem has joined #openstack-infra | 13:15 | |
| AJaeger | wendar: it's the right approach in general | 13:15 |
| AJaeger | chandankumar: see the log file at http://logs.openstack.org/96/498796/2/check/gate-project-config-irc-access/fd763ac/console.html#_2017-08-29_12_56_19_638150 - your sorting is wrong | 13:17 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Multinode jobs should not build images https://review.openstack.org/498772 | 13:18 |
| *** ihrachys has joined #openstack-infra | 13:18 | |
| *** links has quit IRC | 13:18 | |
| *** dave-mccowan has quit IRC | 13:19 | |
| *** jpena|lunch is now known as jpena | 13:20 | |
| AJaeger | wendar: see http://logs.openstack.org/07/498807/1/check/gate-project-config-layout/a09f55f/console.html#_2017-08-29_13_20_30_022243 - those jobs need to get removed as well | 13:24 |
| *** lucasagomes is now known as lucas-brb | 13:26 | |
| openstackgerrit | Merged openstack-infra/shade master: De-client-ify Service Update https://review.openstack.org/482196 | 13:27 |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Add new repo for aodh tempest plugin https://review.openstack.org/498796 | 13:27 |
| wendar | AJaeger: those are the mutexes in zuul/layout.yml | 13:27 |
| wendar | AJaeger: I checked to see if they were used anywhere, and it seemed to only be used on the repos I switched to noop-jobs | 13:28 |
| AJaeger | wendar: let me checkout your change... | 13:28 |
| wendar | AJaeger: I'll remove the mutexes too | 13:28 |
| AJaeger | wendar: yes, let'S do that | 13:28 |
| *** slaweq has quit IRC | 13:29 | |
| *** slaweq has joined #openstack-infra | 13:30 | |
| *** slaweq has quit IRC | 13:30 | |
| *** udesale has quit IRC | 13:30 | |
| openstackgerrit | Allison Randal proposed openstack-infra/project-config master: Remove Packaging Deb project from gate https://review.openstack.org/498807 | 13:31 |
| wendar | AJaeger: updated | 13:31 |
| *** armax has quit IRC | 13:31 | |
| *** slaweq has joined #openstack-infra | 13:32 | |
| *** gouthamr has joined #openstack-infra | 13:33 | |
| *** baoli has joined #openstack-infra | 13:33 | |
| *** ykarel has joined #openstack-infra | 13:34 | |
| *** baoli has quit IRC | 13:34 | |
| *** baoli has joined #openstack-infra | 13:35 | |
| *** jamesmcarthur has joined #openstack-infra | 13:39 | |
| *** dave-mccowan has joined #openstack-infra | 13:41 | |
| *** eharney has joined #openstack-infra | 13:41 | |
| *** ykarel has quit IRC | 13:42 | |
| wendar | AJaeger: okay, passed this time | 13:48 |
| *** slaweq_ has joined #openstack-infra | 13:49 | |
| *** yamamoto has joined #openstack-infra | 13:50 | |
| *** slaweq_ has quit IRC | 13:54 | |
| *** yamamoto has quit IRC | 13:56 | |
| pabelanger | wendar: AJaeger: +3 | 13:57 |
| *** ykarel has joined #openstack-infra | 13:57 | |
| AJaeger | wendar: you're lucky today ;) | 13:57 |
| wendar | AJaeger: happy coincidence on timing :) | 13:57 |
| *** hichihara has joined #openstack-infra | 13:57 | |
| *** mriedem has quit IRC | 13:58 | |
| *** marst has joined #openstack-infra | 14:03 | |
| *** zhurong has quit IRC | 14:03 | |
| *** alexchadin has quit IRC | 14:04 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove Packaging Deb project from gate https://review.openstack.org/498807 | 14:06 |
| *** ociuhandu has quit IRC | 14:07 | |
| *** alexchadin has joined #openstack-infra | 14:08 | |
| *** armax has joined #openstack-infra | 14:08 | |
| *** srobert has joined #openstack-infra | 14:12 | |
| wendar | AJaeger/pabelanger: I put in https://review.openstack.org/498823 with Workflow -1, to get some comments on whether that's the right change to repeat across all the repos. | 14:13 |
| *** ijw has joined #openstack-infra | 14:13 | |
| wendar | AJaeger/pabelanger: There's no real rush on these, and the existing pkgdeb core reviewers can handle the approvals. But, would like your comments on the first one before I copy it across 100+ other repos. :) | 14:15 |
| *** marst has quit IRC | 14:16 | |
| openstackgerrit | Merged openstack-infra/shade master: Support to get resource by id https://review.openstack.org/492080 | 14:17 |
| *** hongbin has joined #openstack-infra | 14:19 | |
| wendar | AJaeger/pabelanger: The Jenkins failure that popped up before I managed to slap on Workflow -1 just shows that the change to noop-jobs hasn't propagated to the gate yet. | 14:19 |
| *** marst has joined #openstack-infra | 14:23 | |
| *** marst_ has joined #openstack-infra | 14:23 | |
| *** rbrndt has joined #openstack-infra | 14:25 | |
| pabelanger | wendar: wfm! | 14:26 |
| wendar | pabelanger: coolness :) | 14:26 |
| *** felipemonteiro__ has quit IRC | 14:26 | |
| *** felipemonteiro__ has joined #openstack-infra | 14:26 | |
| *** marst has quit IRC | 14:27 | |
| *** ijw has quit IRC | 14:28 | |
| *** sree has quit IRC | 14:28 | |
| *** mriedem has joined #openstack-infra | 14:28 | |
| *** sree has joined #openstack-infra | 14:29 | |
| *** baoli has quit IRC | 14:29 | |
| *** annegentle has joined #openstack-infra | 14:30 | |
| *** felipemonteiro__ has quit IRC | 14:31 | |
| *** dprince has joined #openstack-infra | 14:32 | |
| *** sree has quit IRC | 14:33 | |
| *** annegentle has quit IRC | 14:34 | |
| *** eharney has quit IRC | 14:35 | |
| *** annegentle has joined #openstack-infra | 14:37 | |
| *** lucas-brb is now known as lucasagomes | 14:41 | |
| *** chlong has joined #openstack-infra | 14:41 | |
| *** hichihara has quit IRC | 14:42 | |
| *** slaweq_ has joined #openstack-infra | 14:50 | |
| *** Goneri has joined #openstack-infra | 14:50 | |
| *** eharney has joined #openstack-infra | 14:50 | |
| *** alexchadin has quit IRC | 14:52 | |
| *** yamamoto has joined #openstack-infra | 14:52 | |
| *** eumel8 has quit IRC | 14:53 | |
| *** slaweq_ has quit IRC | 14:55 | |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: Make barbican-tempest-plugin voting in barbican https://review.openstack.org/498832 | 14:57 |
| openstackgerrit | Merged openstack-infra/system-config master: Document signing key cross-signing transition https://review.openstack.org/449729 | 14:58 |
| *** yamamoto has quit IRC | 14:58 | |
| openstackgerrit | Merged openstack-infra/system-config master: Silence hieraedit output during key replacement https://review.openstack.org/449730 | 14:59 |
| *** camunoz has joined #openstack-infra | 14:59 | |
| openstackgerrit | Merged openstack-infra/system-config master: Stop saying we announce signing key transitions https://review.openstack.org/449731 | 14:59 |
| *** martinkopec has quit IRC | 15:00 | |
| *** dhajare has quit IRC | 15:01 | |
| *** yamahata has joined #openstack-infra | 15:02 | |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr master: Switch to stestr under the covers https://review.openstack.org/488441 | 15:04 |
| openstackgerrit | Merged openstack-infra/system-config master: Correct signing node double-check example https://review.openstack.org/449732 | 15:04 |
| AJaeger | wendar: LGTM, and zuul is merged, so let's recheck | 15:06 |
| *** baoli has joined #openstack-infra | 15:06 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add publish-openstack-python-docs jobs https://review.openstack.org/498621 | 15:08 |
| wendar | AJaeger: ok, thanks. And, I'll recheck | 15:09 |
| wendar | AJaeger: ah, you already did, and success, great | 15:10 |
| AJaeger | wendar: I did already - and it passed ;) | 15:10 |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Add gate-monasca-api-docs-ubuntu-xenial job https://review.openstack.org/498837 | 15:11 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack-infra/project-config master: Increase Kolla timeout on jobs https://review.openstack.org/498550 | 15:11 |
| AJaeger | wendar: so, good luck with scripting that for your 345 repos - and then merging those and abandoning all open reviews... | 15:12 |
| *** lennyb has quit IRC | 15:12 | |
| *** trown is now known as trown|brb | 15:12 | |
| wendar | AJaeger: aye, fun times ahead. I'll get back to you and pabelanger when I'm ready for Step 3 (possibly during the PTG, if we make good time) | 15:13 |
| *** felipemonteiro has joined #openstack-infra | 15:14 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Add gate-monasca-api-docs-ubuntu-xenial job https://review.openstack.org/498837 | 15:14 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: WIP: Add publish-openstack-python-docs to post pipeline https://review.openstack.org/498840 | 15:14 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: WIP: Add publish-openstack-python-docs to post pipeline https://review.openstack.org/498840 | 15:17 |
| *** felipemonteiro has quit IRC | 15:19 | |
| *** martinkopec has joined #openstack-infra | 15:21 | |
| *** gmann has quit IRC | 15:22 | |
| *** gmann has joined #openstack-infra | 15:22 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add gearman server port configuration https://review.openstack.org/498753 | 15:23 |
| *** lennyb has joined #openstack-infra | 15:23 | |
| *** martinkopec has quit IRC | 15:25 | |
| *** skelso has quit IRC | 15:26 | |
| *** skelso has joined #openstack-infra | 15:26 | |
| *** trown|brb is now known as trown | 15:28 | |
| *** lucasagomes is now known as lucas-afk | 15:28 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Run post playbooks on job timeout https://review.openstack.org/498270 | 15:30 |
| *** rlandy is now known as rlandy|mtg | 15:33 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Delete .pypirc file at end of task https://review.openstack.org/498843 | 15:36 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Move .pypirc into tmpfs https://review.openstack.org/498844 | 15:36 |
| *** iyamahat has joined #openstack-infra | 15:37 | |
| *** slaweq has quit IRC | 15:38 | |
| clarkb | pabelanger: ^ worth noting that /tmp (mktemp's default) isn't necessarily a tmpfs | 15:39 |
| *** iyamahat has quit IRC | 15:39 | |
| pabelanger | clarkb: ya, I'm assuming that is called inside bubblewrap on zuul-executor | 15:39 |
| *** iyamahat has joined #openstack-infra | 15:39 | |
| pabelanger | http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/driver/bubblewrap/__init__.py?h=feature/zuulv3#n188 | 15:40 |
| pabelanger | we default /tmp to --tmpfs today | 15:40 |
| clarkb | ah ok | 15:40 |
| *** mwarad has joined #openstack-infra | 15:41 | |
| *** e0ne has quit IRC | 15:42 | |
| *** andreas_s has quit IRC | 15:43 | |
| *** rama_y has quit IRC | 15:44 | |
| *** martinkopec has joined #openstack-infra | 15:44 | |
| *** mwarad has quit IRC | 15:45 | |
| *** sree has joined #openstack-infra | 15:47 | |
| *** martinkopec has quit IRC | 15:47 | |
| clarkb | dmsimard: I'm going to push up a noop change to devstack-gate that we can compare multinode dvr job logs against | 15:48 |
| dmsimard | clarkb: there are plenty of "no-op" (at least in the context of dvr) job logs available already | 15:49 |
| *** Apoorva has joined #openstack-infra | 15:49 | |
| clarkb | dmsimard: aren't most of them going to be from changes to neutron? I'm not sure I can evaluate what is safe for neutron or not | 15:50 |
| dmsimard | like https://review.openstack.org/#/c/498559/ https://review.openstack.org/#/c/497699/ or https://review.openstack.org/#/c/497618/ | 15:50 |
| pabelanger | 2017-08-29 15:08:51,880 DEBUG is the first time Job openstack-doc-build not defined happend in debug.log today | 15:50 |
| dmsimard | clarkb: oh, I'm just saying the job is already failing on a regular basis across different reviews -- regardless if we're touching the inventory or the overlay bits | 15:50 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 15:51 |
| clarkb | dmsimard: gotcha | 15:51 |
| dmsimard | clarkb: we had switched from the previous dvr job to this new one because it failed less, I haven't gone back to look at the graphs this week | 15:51 |
| clarkb | dmsimard: so the thing to do may be this, we can check experimental to get the normal dvr multinode job (because its a little simpler than grenade) then hold the test nodes and quickly just check the overlay is functioning as expected | 15:51 |
| *** slaweq has joined #openstack-infra | 15:51 | |
| *** felipemonteiro has joined #openstack-infra | 15:52 | |
| *** jascott1 has joined #openstack-infra | 15:52 | |
| clarkb | and ignore dvr | 15:52 |
| *** sree has quit IRC | 15:52 | |
| dmsimard | clarkb: well, the first full tempest passes | 15:52 |
| dmsimard | it's after the upgrade (grenade) that it fails | 15:52 |
| dmsimard | so IMO the overlay was setup properly, right ? | 15:52 |
| *** sree has joined #openstack-infra | 15:52 | |
| clarkb | maybe? I don't know if the first tempest pass in grenade ssh's to anything | 15:52 |
| *** sree has quit IRC | 15:53 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add integration test for zuul_stream https://review.openstack.org/498209 | 15:53 |
| *** sree has joined #openstack-infra | 15:53 | |
| dmsimard | clarkb: ah, wait -- the tempest pass was probably in the inventory patch | 15:53 |
| dmsimard | the network overlay patch there's a failure: http://logs.openstack.org/33/435933/48/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/15da20b/logs/old/testr_results.html.gz | 15:53 |
| *** dtantsur|busy is now known as dtantsur|afk | 15:53 | |
| clarkb | and that is failing to ssh into the instance | 15:54 |
| clarkb | so it could be the overlay or it could be dvr | 15:54 |
| *** yamamoto has joined #openstack-infra | 15:54 | |
| clarkb | probably the simplest thing to do is just to hold the test nodes and check the networking directly | 15:54 |
| *** sree has quit IRC | 15:55 | |
| mordred | pabelanger: +3 on the pypirc patches - however, I think we should make a note to go back through our "delete this file" patches and put them into an always: section on a block so that they clean up even on errors | 15:56 |
| *** slaweq has quit IRC | 15:56 | |
| *** Apoorva has quit IRC | 15:56 | |
| *** jascott1 has quit IRC | 15:56 | |
| openstackgerrit | Ruby Loo proposed openstack-infra/reviewstats master: Update ironic subprojects https://review.openstack.org/498849 | 15:56 |
| clarkb | dmsimard: I've check experimental'd it again. Will hold the nodes that the non grenade dvr multinode job runs on | 15:56 |
| *** Apoorva has joined #openstack-infra | 15:56 | |
| mordred | pabelanger: or alternately to put the cleanups into a handler and then put in an always block that triggers flush handlers: http://docs.ansible.com/ansible/latest/playbooks_blocks.html#id3 | 15:56 |
| pabelanger | mordred: great idea! | 15:56 |
| dmsimard | clarkb: porque no los dos ? | 15:57 |
| *** vhosakot has joined #openstack-infra | 15:57 | |
| pabelanger | mordred: I'll propose a patch here in a few minutes | 15:57 |
| clarkb | dmsimard: mostly because grenade is much more complicated but we can hold them all | 15:57 |
| clarkb | will give us more data /me rechecks too | 15:57 |
| *** yamamoto has quit IRC | 16:00 | |
| *** Apoorva has quit IRC | 16:01 | |
| *** rlandy|mtg is now known as rlandy | 16:01 | |
| *** thorst_afk has quit IRC | 16:01 | |
| *** skelso has quit IRC | 16:02 | |
| *** bh526r has quit IRC | 16:03 | |
| *** skelso has joined #openstack-infra | 16:03 | |
| *** iyamahat_ has joined #openstack-infra | 16:04 | |
| *** thorst_afk has joined #openstack-infra | 16:05 | |
| *** egonzalez has quit IRC | 16:06 | |
| clarkb | dmsimard: ok I've got both test envs held, is there an ssh key I can put in place on the nodes for you? | 16:06 |
| *** iyamahat has quit IRC | 16:08 | |
| *** iyamahat__ has joined #openstack-infra | 16:08 | |
| *** ijw has joined #openstack-infra | 16:09 | |
| *** ijw has joined #openstack-infra | 16:09 | |
| *** iyamahat_ has quit IRC | 16:10 | |
| otherwiseguy | anyone have any idea what is going on here: http://logs.openstack.org/periodic/periodic-neutron-py35-with-ovsdbapp-master/279914f/console.html#_2017-08-29_06_27_21_835926 (tox.ConfigError: ConfigError: substitution key '[py35]setenv' not found) on this periodic job? | 16:11 |
| dmsimard | clarkb: so... | 16:12 |
| dmsimard | clarkb: looking at the graphs, we probably cursed the grenade dvr job because as soon as we started using it, it went in near 100% failure | 16:12 |
| dmsimard | last 30 days http://i.imgur.com/FCA3GY7.png | 16:13 |
| clarkb | dmsimard: nice | 16:13 |
| dmsimard | clarkb: this is single node dvr: http://i.imgur.com/0X8M844.png and multinode dvr: http://i.imgur.com/VAd4Eqo.png | 16:14 |
| dmsimard | ihrachys: ^ fyi | 16:14 |
| *** iyamahat__ has quit IRC | 16:14 | |
| dmsimard | ihrachys: is there anyone working on the multinode dvr job stability yet ? | 16:14 |
| *** iyamahat__ has joined #openstack-infra | 16:14 | |
| clarkb | dmsimard: my guess is that job's troubles are more likely related to the release than running it against devstack-gate | 16:15 |
| clarkb | dmsimard: its possible that neutron can't actually safely upgrade a dvr install | 16:15 |
| *** tesseract has quit IRC | 16:15 | |
| pabelanger | clarkb: so, for some odd reason, infracloud-vanilla mirror fails way more then infracloud-chocolate mirror to buildlogs.centos. Its possible we are scheduling more centos jobs to that cloud, but I am also starting to wonder if there is something wrong specific to that mirror. | 16:15 |
| pabelanger | clarkb: AFAIK, both have the same networking | 16:15 |
| dmsimard | clarkb: hm, maybe. But even the regular multi node dvr is basically 100% failure | 16:15 |
| ihrachys | dmsimard, not that I am aware | 16:15 |
| clarkb | pabelanger: doesn't vanilla run more jobs? | 16:15 |
| openstackgerrit | Ruby Loo proposed openstack-infra/reviewstats master: ironic subprojects are updated https://review.openstack.org/498849 | 16:16 |
| dmsimard | ihrachys: It's hard to figure out if our changes to network overlay out of devstack-gate are having a negative impact when jobs testing multi-node DVR (both regular tempest and grenade) are near-100% failure rates. Any tips ? | 16:16 |
| pabelanger | clarkb: it does, 10 more nodes then chocolate. https://review.openstack.org/498603/ makes them both 41 | 16:17 |
| dmsimard | ihrachys: We're currently going to hold nodes to manually check if the bridge works as we expect it to | 16:17 |
| *** Guest84763 has quit IRC | 16:17 | |
| dmsimard | ihrachys: been using http://grafana.openstack.org/dashboard/db/neutron-failure-rate as a reference for failure rates | 16:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add X-Github-Delivery id to debug logs https://review.openstack.org/498853 | 16:18 |
| *** rkukura_ has joined #openstack-infra | 16:18 | |
| clarkb | pabelanger: why don't we start with equivalent node quotas then check if the failures continue to skew one way or the other? | 16:19 |
| pabelanger | clarkb: sure, I'm good with that. | 16:20 |
| clarkb | especially since chocolate was basically offline for a good chunk of yesterday | 16:20 |
| clarkb | so the historical data is a bit dirty | 16:20 |
| pabelanger | clarkb: I've also started discussions in #centos-devel to see if we can logs from their side | 16:20 |
| clarkb | cool | 16:20 |
| fungi | #status log replaced myself with clarkb at https://review.openstack.org/#/admin/groups/infra-ptl | 16:20 |
| openstackstatus | fungi: finished logging | 16:20 |
| fungi | clarkb: lmk if there's anywhere else i've missed | 16:20 |
| clarkb | fungi: does that mean I have to approve infra-specs changes now? | 16:21 |
| fungi | you don't _have_ to, but nobody else is going to! | 16:21 |
| *** rkukura has quit IRC | 16:21 | |
| *** rkukura_ is now known as rkukura | 16:21 | |
| *** sambetts is now known as sambetts|afk | 16:21 | |
| *** iyamahat__ has quit IRC | 16:23 | |
| *** annegentle has quit IRC | 16:23 | |
| *** iyamahat__ has joined #openstack-infra | 16:23 | |
| openstackgerrit | Major Hayden proposed openstack-infra/project-config master: Add nv CentOS OSA-tempest job https://review.openstack.org/498857 | 16:24 |
| *** Swami has joined #openstack-infra | 16:26 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Remove JOB_NAME from reproduce https://review.openstack.org/498559 | 16:26 |
| clarkb | otherwiseguy: I think that is abusing some fancy feature in tox where py35-foo means run with py35 environment options and its failing to find those? I'd have to go read the docs to figure out what the intent is there | 16:27 |
| clarkb | otherwiseguy: it definitely seems to be trying to find the py35 setenv value though | 16:27 |
| *** thorst_a_ has joined #openstack-infra | 16:28 | |
| openstackgerrit | Merged openstack-infra/project-config master: Bump infracloud-chocolate by 10 servers https://review.openstack.org/498603 | 16:29 |
| *** ykarel has quit IRC | 16:30 | |
| *** LindaWang has quit IRC | 16:31 | |
| *** thorst_afk has quit IRC | 16:32 | |
| fungi | clarkb: jeblair: as a followup to the intel management engine vulnerability a little while back, this is a rather interesting discovery: http://blog.ptsecurity.com/2017/08/disabling-intel-me.html | 16:32 |
| clarkb | otherwiseguy: has that job ever passed. THe more I am reading the neutron tox.ini the more I think you just want tox -epy35, the suffix isn't used anywhere in tox.ini. http://tox.readthedocs.io/en/latest/config.html#generating-environments-conditional-settings for the docs on this | 16:32 |
| clarkb | otherwiseguy: but I think that is why it fails? neutron-master isn't used anywhere? | 16:33 |
| fungi | setting aside my tinfoil cap, it's at least amusing that the united states government had them implement a kill switch for that, and then intel failed to disclose that when claiming there was no way to disable | 16:33 |
| persia | Many sovereign contracts include particularly stongly binding confidentiality clauses, so it may not even be that the folk making the claim were aware (trying to be an apologist, not a conspiracist). | 16:35 |
| fungi | persia: yep, completely agree | 16:35 |
| clarkb | also its totally not a supported configuration | 16:36 |
| fungi | of course not! | 16:36 |
| *** dhajare has joined #openstack-infra | 16:37 | |
| fungi | if you were coerced into implementing a remote backdoor, you certainly couldn't officially/publicly support anything to effectively disable it | 16:37 |
| *** jpich has quit IRC | 16:38 | |
| fungi | not that i think that's what happened, but it sure does look bad for them pr-wise | 16:38 |
| otherwiseguy | clarkb, yeah, that test has never passed. i just did some tests with a locally generated and it just seems like since [py35] which is autogenerated doesn't contain setenv, it fails? | 16:38 |
| otherwiseguy | And apparently I wrote the script that is failing and completely forgot that. :p | 16:39 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add X-Github-Delivery id to debug logs https://review.openstack.org/498853 | 16:39 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Write a logging config file and pass it to callbacks https://review.openstack.org/498127 | 16:39 |
| otherwiseguy | Getting old sucks. | 16:39 |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Add release-announce job https://review.openstack.org/498623 | 16:39 |
| clarkb | otherwiseguy: ya I think thats the cause. You can probably confirm it by editing tox.ini locally and running that tox command | 16:39 |
| jlvillal | How is 0.9.7 the latest version of hacking, if there exists 0.13.0? https://pypi.python.org/pypi/hacking/0.13.0 | 16:39 |
| clarkb | should at least get far enough along to install packages and such if its fixed | 16:39 |
| jlvillal | Am I just not understanding version numbers? | 16:39 |
| fungi | jlvillal: where is 0.9.7 indicated to be the latest version of hacking? | 16:39 |
| clarkb | jlvillal: what says 0.9.7 is the latest? | 16:40 |
| jlvillal | when you click that link | 16:40 |
| jlvillal | or https://pypi.python.org/pypi/hacking | 16:40 |
| fungi | jlvillal: oh, what you're not understanding is pypi's confusing behavior of saying "latest" to mean "most recently uploaded" | 16:40 |
| *** efried has quit IRC | 16:40 | |
| clarkb | pip installs 0.13.0 properly as biggest version | 16:41 |
| fungi | it's not at all sane when you release from multiple branches/series | 16:41 |
| jlvillal | fungi: Oh and :( | 16:41 |
| jlvillal | fungi: clarkb: Thanks | 16:41 |
| clarkb | otherwiseguy: as an alternative just drop the extra venv bits, and just run py35 instead | 16:41 |
| otherwiseguy | clarkb, yep, that script is just broken for passing in things like py35 | 16:41 |
| *** felipemonteiro has quit IRC | 16:41 | |
| otherwiseguy | adding a [py35]setenv=BOB=1 basepython=3 etc. makes it work locally. | 16:42 |
| fungi | jlvillal: we've discussed some (suboptimal) ways to make that less confusing but nobody's really run with one yet | 16:43 |
| otherwiseguy | this is what happens when I do something in infra code :p | 16:43 |
| *** rama_y has joined #openstack-infra | 16:43 | |
| jlvillal | fungi: Yeah, and I so far have not been able to see the list of all the versions available via the web page | 16:43 |
| fungi | jlvillal: you can find them via the pypi "simple" api | 16:43 |
| clarkb | https://pypi.python.org/simple/hacking | 16:43 |
| fungi | jlvillal: https://pypi.python.org/simple/hacking/ | 16:44 |
| jlvillal | fungi: Thanks :) | 16:44 |
| fungi | also marvellously not sorted (or sorted in some way i'm unable to suss out) | 16:44 |
| otherwiseguy | clarkb, so probably just delete the tox.ini generation and run $script_path/run-tox.sh $venv? | 16:44 |
| fungi | jlvillal: showing only the "latest" (most recent) upload and hiding the rest is a toggle we can set in pypi, if memory serves. we can also get fancy and work out some way to determine which releases to hide via external scripting. might be awesome, or might be fragile | 16:45 |
| *** jamesmcarthur has quit IRC | 16:46 | |
| fungi | originally we defaulted to hiding "old" releases so that people going to a link like https://pypi.python.org/pypi/hacking/ wouldn't end up presented with links to an insanely long list of version numbers | 16:46 |
| *** jamesmcarthur has joined #openstack-infra | 16:46 | |
| *** rcernin has quit IRC | 16:46 | |
| clarkb | otherwiseguy: basically since you are just inheriting everything from py35 and not overriding anything I thnik you can drop the modification to tox.ini and invoke tox -epy35 instead | 16:46 |
| clarkb | otherwiseguy: at the end of that script | 16:46 |
| otherwiseguy | clarkb, ok, will do. ovsdbapp is the only thing that uses the script. so changing can only hurt me. ;D | 16:47 |
| fungi | jlvillal: once pypi.org (warehouse) has officially replaced pypi.python.org (cheeseshop) the difference in interface is probably a great reason to revisit that decision anyway | 16:47 |
| fungi | jlvillal: also, https://pypi.org/project/hacking/ shows 0.13.0 by default it looks like | 16:48 |
| fungi | so its definition of "latest version" seems to be saner than in the cheeseshop | 16:48 |
| *** pcaruana has quit IRC | 16:48 | |
| jlvillal | fungi: thanks! | 16:49 |
| fungi | and then you can get to the earlier ones via https://pypi.org/project/hacking/#history | 16:49 |
| openstackgerrit | Eric Harney proposed openstack-infra/project-config master: Non-voting unit test job for cinder driverfixes https://review.openstack.org/498868 | 16:49 |
| *** dave-mccowan has quit IRC | 16:50 | |
| fungi | nicely sorted in descending order by version number, with dates mentioned for reference | 16:50 |
| *** thorst_a_ has quit IRC | 16:51 | |
| openstackgerrit | Terry Wilson proposed openstack-infra/project-config master: Remove broken tox.ini generation from script https://review.openstack.org/498870 | 16:51 |
| *** thorst_afk has joined #openstack-infra | 16:52 | |
| *** slaweq has joined #openstack-infra | 16:52 | |
| *** iyamahat__ has quit IRC | 16:52 | |
| *** efried has joined #openstack-infra | 16:53 | |
| *** links has joined #openstack-infra | 16:55 | |
| *** dhajare has quit IRC | 16:55 | |
| *** thorst_afk has quit IRC | 16:56 | |
| *** yamamoto has joined #openstack-infra | 16:56 | |
| *** slaweq has quit IRC | 16:57 | |
| *** electrofelix has quit IRC | 16:57 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Update config cache only after all cat jobs complete https://review.openstack.org/498872 | 16:57 |
| *** jpena is now known as jpena|off | 16:59 | |
| *** iyamahat has joined #openstack-infra | 17:00 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Fix badly named function typo https://review.openstack.org/498877 | 17:00 |
| *** baoli has quit IRC | 17:01 | |
| *** baoli has joined #openstack-infra | 17:02 | |
| *** sree has joined #openstack-infra | 17:02 | |
| *** yamamoto has quit IRC | 17:02 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/reviewstats master: WIP/DNM: Testing https://review.openstack.org/498878 | 17:03 |
| *** ralonsoh has quit IRC | 17:03 | |
| *** thorst_afk has joined #openstack-infra | 17:04 | |
| *** sree has quit IRC | 17:06 | |
| *** hashar is now known as hasharDinner | 17:06 | |
| *** ramishra has quit IRC | 17:10 | |
| pabelanger | clarkb: fungi: any objections to promote 498772 in the gate? that will help stop the gate resetting for tripleo, which is backed up to 16hr+ today. Fixed bug 1713167 http://status.openstack.org/elastic-recheck/#1713167 | 17:10 |
| openstack | bug 1713167 in tripleo "build-images : Throw error msg if image build is failing *** Image Build Error ***" [High,Triaged] https://launchpad.net/bugs/1713167 | 17:10 |
| *** raissa has quit IRC | 17:10 | |
| fungi | it is impacting puppet-openstack changes too i guess, so makes sense to go ahead and get bumped up | 17:11 |
| pabelanger | k, promoting change now | 17:11 |
| fungi | the bug could stand a mention of the change fixing it | 17:12 |
| pabelanger | fungi: thanks for updating | 17:13 |
| *** baoli has quit IRC | 17:14 | |
| *** slaweq has joined #openstack-infra | 17:14 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Limit github PR search to project status is from https://review.openstack.org/498879 | 17:14 |
| openstackgerrit | Merged openstack-infra/project-config master: Increase Kolla timeout on jobs https://review.openstack.org/498550 | 17:14 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove broken tox.ini generation from script https://review.openstack.org/498870 | 17:16 |
| clarkb | seems weird that we would delete the check rather than updating the check | 17:18 |
| *** efoley has quit IRC | 17:18 | |
| clarkb | I guess because there is never a case where you'd build an image? | 17:18 |
| *** rbrndt has quit IRC | 17:21 | |
| *** ijw has quit IRC | 17:22 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Limit github PR search to project status is from https://review.openstack.org/498879 | 17:22 |
| otherwiseguy | clarkb, is there any way to manually trigger a periodic test run? | 17:23 |
| clarkb | otherwiseguy: a zuul admin can directly enqueue the job, otherwise no. In many cases they have corresponding jobs in check/experimental for this reason | 17:25 |
| *** kjackal has quit IRC | 17:26 | |
| *** trown is now known as trown|lunch | 17:26 | |
| *** kjackal has joined #openstack-infra | 17:27 | |
| *** baoli has joined #openstack-infra | 17:29 | |
| *** Shrews has quit IRC | 17:33 | |
| *** yamahata has quit IRC | 17:33 | |
| pabelanger | clarkb: fungi: do we know the regional location of infracloud, is that austin, texas? | 17:33 |
| *** Shrews has joined #openstack-infra | 17:34 | |
| *** iyamahat has quit IRC | 17:34 | |
| clarkb | houston | 17:34 |
| *** links has quit IRC | 17:34 | |
| fungi | hopefully on high ground | 17:36 |
| mordred | I believe they moved it to high ground the last time there was a flood | 17:36 |
| inc0 | pabelanger AJaeger thanks for reviews, yeah no argument there, I want gates to be as fast as possible too | 17:36 |
| *** slaweq has quit IRC | 17:36 | |
| fungi | granted, this is a bigger flood, might require still higher ground | 17:36 |
| clarkb | dmsimard: I've just commented on the overlay change. Being able to hop on the held nodes helped. If you want to see things first hand just give me an ssh key and I will set that up for you | 17:37 |
| inc0 | I was part of fastest datacenter migration in human history (I think...) | 17:37 |
| inc0 | we just opened new dc and flood hit old dc | 17:38 |
| inc0 | literally same week | 17:38 |
| inc0 | suffice to say we prioritised servers based on rack position | 17:38 |
| clarkb | dmsimard: mordred also where are we logging this? | 17:38 |
| clarkb | dmsimard: mordred its possible grenade just fails at logging it due to its weird setup, but would be good to make sure we are logging basic steps taken here | 17:40 |
| clarkb | ok we set -x | 17:42 |
| clarkb | so we should get the ansible output /me looks | 17:42 |
| clarkb | dmsimard: mordred looks like it logs that we run the ansible tasks but not any output, which I guess is a good thing? Really what we need is the equivalent of bash set -x while debugging this. separately is http://logs.openstack.org/33/435933/48/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/743152b/console.html#_2017-08-29_16_15_05_046624 facts? we probably can remove that right? | 17:45 |
| mordred | clarkb: no - that's the return value from using the service module to start the service I believe | 17:48 |
| mordred | clarkb: I see a bunch of output there (albeit not super-awesome formatting) - can you point me to a place where you see task but no output? | 17:49 |
| *** eharney has quit IRC | 17:49 | |
| clarkb | mordred: http://logs.openstack.org/33/435933/48/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/743152b/console.html#_2017-08-29_16_15_07_106141 is specifically the one that is "failing" I mean its succeeding but we passed it the wrong arguments and it is hard to see that there | 17:50 |
| pabelanger | clarkb: fungi: so, quick chat with centos-devel about infracloud-vanilla, it seems buildlogs.centos.org has 4 servers behind it, 2 in .us, 2 in .eu. When I checked DNS on infracloud-vanilla, it was pointing to .eu server. So it is possible, something in the geolocation dns lookup code is not correct. Next step is to create bug on centos-devel issue tracker and move forward. | 17:50 |
| clarkb | pabelanger: nice | 17:50 |
| mordred | pabelanger: ++ | 17:50 |
| clarkb | going from houston to eu definitely seems less than ideal :) | 17:50 |
| mordred | uhm, yeah | 17:50 |
| pabelanger | clarkb: fungi: I did manage to find out one of the server IP address, I'm also considering adding it to /etc/hosts on infracloud-vanilla and see if that helps for a few hours | 17:51 |
| clarkb | mordred: basically we are creating a vxlan tunnel with the wrong IP endpoints | 17:51 |
| clarkb | mordred: so seeing what we are executing just a bit more would be useful | 17:51 |
| *** iyamahat has joined #openstack-infra | 17:51 | |
| *** iyamahat has quit IRC | 17:51 | |
| clarkb | (kind of like bash set -x tracing) | 17:52 |
| mordred | clarkb: nod - what should I search for in that log to see the invocation we do have? | 17:52 |
| *** iyamahat has joined #openstack-infra | 17:52 | |
| clarkb | mordred: well I don't think we log it at all | 17:52 |
| pabelanger | 38.110.33.4 is 1 of the .us servers | 17:52 |
| clarkb | mordred: I figured it out by holding nodes and checking the results | 17:52 |
| mordred | ah. gotcha | 17:52 |
| pabelanger | 163.172.17.132 is the .eu | 17:52 |
| mordred | clarkb: can youpoint me to either the source code or tell me what to look at on the held node? | 17:53 |
| clarkb | mordred: yes, https://review.openstack.org/#/c/435933/48/playbooks/roles/ovs_vxlan_bridge_peers/tasks/main.yaml where I have commented is the task and you can see the results of that on 104.130.163.176 running `ovs-vsctl show` and looking under br_ex | 17:53 |
| clarkb | mordred: you'll see the local and remote IPs are identical | 17:54 |
| mordred | clarkb: ah - gotcha | 17:54 |
| pabelanger | clarkb: fungi: rolling1.centos.org and rolling2.centos.org appear to be .us servers, and rolling3 / rolling4 .eu | 17:54 |
| clarkb | mordred: thinking that for when people start adding new playbooks/roles that an equivalent to bash -x may be useful. I don't think we'd run with that 24/7 but when developing new things like this would be useful | 17:56 |
| clarkb | maybe thats ansible -vvv? | 17:56 |
| mordred | clarkb: oh - well, the logging situation in v3 is TOTALLY different | 17:56 |
| fungi | pabelanger: could you request additional round-robin names for us and eu so that people who want to skip the geolocation can do so? | 17:56 |
| mordred | clarkb: for now for this job, if you go to http://logs.openstack.org/33/435933/48/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/743152b/logs/ara/ | 17:56 |
| clarkb | mordred: ya that doesn't actually show you the command ,if you click on the command link it errors | 17:57 |
| mordred | clarkb: click on tasks in the ovs_vxlan_bridge.yaml | 17:57 |
| *** slaweq has joined #openstack-infra | 17:57 | |
| mordred | then "changed" on "ovs_vxlan_bridge_peers : Add port to bridge on primary" | 17:57 |
| pabelanger | fungi: possible. I'm going to add that in to the bug repo | 17:57 |
| clarkb | oh it actually worked this time clicking on command | 17:57 |
| mordred | you can get this: | 17:57 |
| mordred | http://logs.openstack.org/33/435933/48/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/743152b/logs/ara/result/2cbd7ffd-7145-425d-be5e-89a24e68e30d/ | 17:57 |
| *** yamamoto has joined #openstack-infra | 17:58 | |
| clarkb | mordred: ah yup that is useful | 17:58 |
| clarkb | in any case I think this is likely to be the last bits of fixing for overlays. Poking around everything else (except the interface name which I commented on as well) looks correct | 17:58 |
| mordred | clarkb: we have a bunchof choices in the v3 stack, but hopefully in the intervening period the data ara collects can be helpful | 17:58 |
| pabelanger | fungi: we could also make our apache proxypass logic more complex but setting up our own balancer, but that might be a little excessive | 17:58 |
| mordred | clarkb: woot! | 17:58 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/project-config master: Fix playbook path for proposal jobs https://review.openstack.org/498895 | 17:59 |
| pabelanger | #status log /etc/hosts on mirror.regionone.infracloud-vanilla.org has buildlogs.centos.org pinned to 38.110.33.4. This is temporary to see if round robin DNS is our issues when we proxy to buildlogs.centos.org | 18:00 |
| openstackstatus | pabelanger: finished logging | 18:00 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/reviewstats master: WIP/DNM: Testing https://review.openstack.org/498878 | 18:00 |
| *** felipemonteiro has joined #openstack-infra | 18:01 | |
| clarkb | dmsimard: mordred ^fyi the permalink thing doesn't seem to work on firefox | 18:01 |
| clarkb | was realyl confused at how mordred got that link until I tried in chrome | 18:01 |
| *** slaweq has quit IRC | 18:01 | |
| mordred | clarkb: good to know | 18:01 |
| *** baoli has quit IRC | 18:03 | |
| *** yamamoto has quit IRC | 18:04 | |
| *** annegentle has joined #openstack-infra | 18:04 | |
| dmsimard | clarkb: You get the permalink from the blue chain link thing at the top right of the modals. It doesn't work for you ? | 18:05 |
| clarkb | dmsimard: correct, if you click that on firefox nothing happens and right clicking doesn't allow for copying the link either | 18:06 |
| clarkb | works in chrome though | 18:06 |
| clarkb | firefox 54.0.1 64bit build for linux | 18:06 |
| dmsimard | clarkb: odd, let me open up firefox | 18:06 |
| dmsimard | clarkb: huh, you're right. | 18:07 |
| *** baoli has joined #openstack-infra | 18:08 | |
| openstackgerrit | Eric Kao proposed openstack-infra/project-config master: request increased timeout https://review.openstack.org/498897 | 18:08 |
| *** ijw has joined #openstack-infra | 18:09 | |
| *** yamahata has joined #openstack-infra | 18:10 | |
| *** rbrndt has joined #openstack-infra | 18:12 | |
| *** wolverineav has quit IRC | 18:14 | |
| openstackgerrit | Merged openstack-infra/project-config master: Extract scripts copying into a role https://review.openstack.org/498111 | 18:16 |
| *** wolverineav has joined #openstack-infra | 18:16 | |
| *** jascott1 has joined #openstack-infra | 18:18 | |
| dmsimard | clarkb: there's no errors in the javascript console either, ew | 18:19 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add oaktree to v3 project list for readthedocs testing https://review.openstack.org/498899 | 18:21 |
| fungi | you wouldn't think you'd need fancy opaque javascript for a button to take you to a subpage | 18:22 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/reviewstats master: WIP/DNM: Testing https://review.openstack.org/498878 | 18:23 |
| dmsimard | fungi: there's a reason for that | 18:23 |
| fungi | oh, i'm sure there is | 18:23 |
| openstackgerrit | Merged openstack-infra/project-config master: Fix playbook path for proposal jobs https://review.openstack.org/498895 | 18:23 |
| fungi | all in all the interface is really slick. this is the first time i've ever really taken the time to look through it | 18:24 |
| *** pvaneck has joined #openstack-infra | 18:24 | |
| dmsimard | tl;dr, there's thousands of pages in even a moderately sized playbook run (think, one page for each result, pages for host facts, pages for files, etc.) | 18:24 |
| dmsimard | So, instead of having to generate what would end up being thousands of modal popups, there's just one and the content is dynamically loaded | 18:25 |
| fungi | got it, so the subpages are being generated on the fly and aren't actual backing files | 18:25 |
| *** jamesmcarthur has quit IRC | 18:25 | |
| dmsimard | the *content* of the modals are actually embedded within iframes (which are the pages you get when clicking the permalink button) | 18:25 |
| fungi | any reason you don't hyperlink python in the top-level header but link everything else in that line? | 18:26 |
| dmsimard | The other challenge is to keep the amount of pages to generate to a minimum so that static reports are generated quickly and stay relatively lightweight | 18:26 |
| dmsimard | fungi: eh, I dunno, I could link to python.org I guess | 18:27 |
| fungi | seems like that would at least make it consistent with the ansible link | 18:27 |
| *** adisky__ has quit IRC | 18:27 | |
| dmsimard | fungi: want to send a review? :) https://github.com/openstack/ara/blob/18a3ab418fd333a037b897a73b0737e464c5c3bc/ara/templates/navbar.html#L19 | 18:28 |
| fungi | sure, i should be able to remember how that works in gh ;) | 18:29 |
| fungi | oh, nm, that's in gerrit! | 18:29 |
| dmsimard | fungi: er, of course it's in gerrit | 18:29 |
| fungi | i was thrown off by the gh url | 18:30 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make a docs-on-readthedocs project-template https://review.openstack.org/498903 | 18:30 |
| clarkb | dmsimard: have you seen my comments on the overlay change yet? | 18:30 |
| dmsimard | clarkb: not yet, catching up on pings from lunch, getting to it soon | 18:30 |
| *** jamesmcarthur has joined #openstack-infra | 18:32 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add oaktree to v3 project list for readthedocs testing https://review.openstack.org/498899 | 18:32 |
| dmsimard | clarkb: ok cool, sending a patch | 18:33 |
| fungi | dmsimard: https://review.openstack.org/498904 should do it (seems you already had an empty <a> tag so must have been a placeholder you meant to get back around to fleshing out anyway) | 18:34 |
| dmsimard | wfm, and yeah the <a> has a css effect applied to it iirc, that's why it was there but no target | 18:34 |
| * dmsimard sucks in frontend | 18:35 | |
| fungi | heh | 18:35 |
| *** trown|lunch is now known as trown | 18:36 | |
| *** dprince has quit IRC | 18:36 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add oaktree to v3 project list for readthedocs testing https://review.openstack.org/498899 | 18:39 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add pre-release pipeline definition to v3 config https://review.openstack.org/498907 | 18:39 |
| dmsimard | clarkb: re: https://review.openstack.org/#/c/435933/48/playbooks/roles/ovs_vxlan_bridge_peers/tasks/main.yaml | 18:41 |
| dmsimard | what about lines 32 through 38 ? | 18:41 |
| *** lihi has quit IRC | 18:41 | |
| *** slaweq has joined #openstack-infra | 18:42 | |
| *** lihi has joined #openstack-infra | 18:42 | |
| mordred | clarkb, fungi, pabelanger, jeblair: if you don't mind, https://review.openstack.org/#/c/498626 then those two ^^ would be helpful | 18:42 |
| clarkb | dmsimard: 32 to 38 appear to be ok because that is the other half of the tunnel connection. Basically we have a tunnel configured on primary to subnode and on subnode to primary | 18:42 |
| fungi | lookin' | 18:43 |
| clarkb | dmsimard: it appears the subnode setup is correct | 18:43 |
| dmsimard | clarkb: ok, makes sense | 18:43 |
| dmsimard | was making usre | 18:43 |
| dmsimard | sure* | 18:43 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make a docs-on-readthedocs project-template https://review.openstack.org/498903 | 18:43 |
| clarkb | dmsimard: if we had working multicast we'd just tell each one to talk to the proper multicast group and be done | 18:43 |
| dmsimard | btw infra meeting in ~15 minutes in #openstack-meeting, agenda: https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting | 18:43 |
| *** slaweq has quit IRC | 18:44 | |
| clarkb | dmsimard: which is a much simpler setup and what you'd nomrally see if large vxlan use cases but not here because enomulticast | 18:44 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/devstack-gate master: Move network overlay configuration to Ansible playbooks https://review.openstack.org/435933 | 18:47 |
| *** eharney has joined #openstack-infra | 18:49 | |
| fungi | mordred: one concern with 498907 | 18:50 |
| fungi | see inline comment | 18:50 |
| *** aeng has joined #openstack-infra | 18:51 | |
| dmsimard | clarkb: I added a todo to iron out that firefox link issue for 1.0, can't figure it out at a quick glance | 18:51 |
| clarkb | dmsimard: thanks, I wish I was more help but anything to do with web browsers is pretty much magic to me | 18:51 |
| dmsimard | the javascript involved is simple and mostly handwritten | 18:51 |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Add release-announce job and publish-to-pypi template https://review.openstack.org/498623 | 18:51 |
| dmsimard | ¯\_(ツ)_/¯ | 18:51 |
| clarkb | I'm going to hold the dvr multinode job that runs against latest changes so we can double check by hand again | 18:51 |
| clarkb | (since I don't trust that job to tell us based on pass/fail) | 18:51 |
| clarkb | and held | 18:54 |
| Zara | hi, all! a colleague and I are comparing different api versions for some projects; wondering if there's a tool to do this anywhere? we were considering diffing the request/response tables from the docs, though I don't yet know how uniform those will be | 18:55 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul feature/zuulv3: Add integration test for zuul_stream https://review.openstack.org/498209 | 18:55 |
| clarkb | Zara: I think the api docs are version aware. Not sure if it will do diffing though | 18:56 |
| *** camunoz has quit IRC | 18:57 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Remove bindep_command and bindep_fallback references https://review.openstack.org/498913 | 18:57 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add bindep_command and bindep_fallback to site-variables https://review.openstack.org/498914 | 18:58 |
| Zara | yeah, things seem to be prefixed with versions, so I'd guess it would be possible to script it if something doesn't exist already (I guess I'd just have to track down the relevant repos in that case) | 18:58 |
| *** jtomasek_ has quit IRC | 18:59 | |
| *** yamamoto has joined #openstack-infra | 18:59 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Remove bindep_command and bindep_fallback references https://review.openstack.org/498916 | 18:59 |
| *** niska has quit IRC | 18:59 | |
| mordred | fungi: thanks! looking | 19:00 |
| mordred | Zara: api versions for openstack projects? | 19:00 |
| *** eranrom has joined #openstack-infra | 19:01 | |
| Zara | yeah, so any changes in api between say v2 of foo-client and v3 of foo-client | 19:02 |
| Zara | it may be easier to do by eye in the end | 19:03 |
| *** ijw has quit IRC | 19:03 | |
| *** ijw has joined #openstack-infra | 19:04 | |
| mordred | Zara: yah- it turns out to be rather complex | 19:04 |
| *** niska has joined #openstack-infra | 19:04 | |
| *** yamamoto has quit IRC | 19:05 | |
| mordred | Zara: although I _highly_ recommend not using foo-client if you're talking about python clients for openstack and instead to either use shade or just direct REST calls via keystoneauth1.adapter.Adapter - we just added all the needed version discovery stuff to ksa this cycle | 19:06 |
| *** ijw has quit IRC | 19:08 | |
| *** dprince has joined #openstack-infra | 19:09 | |
| Zara | yeah, I'm not sure if I'm referring to python clients or not yet tbf, since I'm not sure if fooclient is the same as python-fooclient in general in the openstack world. | 19:13 |
| *** slaweq has joined #openstack-infra | 19:15 | |
| Zara | (since I'm aware the various python clients are generally being deprecated at least in part in favour of osc) | 19:16 |
| *** Goneri has quit IRC | 19:17 | |
| fungi | Zara: well, the term "api" has been conflated. most services have a "rest api" and then a "python api" which is an importable module that is usually coupled with a (increasingly deprecated) command-line client | 19:17 |
| Zara | yeah, I'm currently assuming most projects do it like storyboard (so api and then python-foo as the importable thing/cli), but I've never actually checked that assumption | 19:18 |
| openstackgerrit | Kendall Nelson proposed openstack-infra/storyboard master: Adds Migration Documentation https://review.openstack.org/496439 | 19:18 |
| dmsimard | Zara: there's some exceptions, like openstackclient and python-openstackclient are two separate things | 19:18 |
| dmsimard | Zara: http://lists.openstack.org/pipermail/openstack-dev/2017-August/121419.html | 19:19 |
| fungi | the command-line clients are giving way to the unified openstackclient (at least for non-admin functionality), and the importable modules which came with them are a lot more confusing to use and lack a fair amount of consistency compared to relying on shade (though still fairly heavily used for service-to-service communication in openstack deployments themselves) | 19:19 |
| *** armax has quit IRC | 19:20 | |
| Zara | ah okay, so as a general scribbly rule, 'for thing interfacing with an openstack deployment = use shade,for within an openstack deployment = you might need the clients'? | 19:21 |
| Zara | dmsimard: thanks for the link | 19:22 |
| fungi | more or less | 19:23 |
| mordred | Zara: yah - or - if you want to interact with openstack things via python, use shade, if you want to interact on the command line, use python-openstackclient | 19:23 |
| fungi | for interfacing with an openstack deployment use shade or the rest apis (and perhaps soon-ish the latter becomes "use oaktree's rest api) | 19:23 |
| *** aeng has quit IRC | 19:24 | |
| fungi | shade is also absorbing the unified python sdks effort, it seems | 19:24 |
| Zara | heh | 19:24 |
| * mordred needs to submit thatpatch - maybe I'll push that up today | 19:24 | |
| Zara | thanks, fungi, mordred | 19:24 |
| Zara | even if I still have no super-fast way to compare the apis, I now at least learned a thing :) | 19:25 |
| mordred | Zara: in any case, for better or worse I know WAY too much about consuming openstack rest apis - so more than happy to help in any way I can | 19:26 |
| Zara | hahaha | 19:26 |
| Zara | thank you :) | 19:26 |
| *** yamamoto has joined #openstack-infra | 19:27 | |
| *** ijw has joined #openstack-infra | 19:27 | |
| *** dprince has quit IRC | 19:32 | |
| *** ijw has quit IRC | 19:32 | |
| *** gcb has quit IRC | 19:33 | |
| *** e0ne has joined #openstack-infra | 19:34 | |
| *** ijw has joined #openstack-infra | 19:34 | |
| *** dave-mccowan has joined #openstack-infra | 19:35 | |
| fungi | comparing rest apis might be possible by leveraging mechanisms similar to the automated api documentation | 19:36 |
| Zara | hm, I wondered about that, then it looked like some of the api guides might actually be handwritten (but maybe projects have multiple ones and I was looking at the wrong ones, or maybe it isn't done for whatever reason, but it doesn't mean that it can't be) | 19:38 |
| openstackgerrit | Slawek Kaplonski proposed openstack-infra/shade master: Fix handling timeouts in volume functional tests cleanup https://review.openstack.org/498918 | 19:38 |
| *** gixxer1k1k has joined #openstack-infra | 19:38 | |
| Zara | (I | 19:38 |
| Zara | erm | 19:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/jeepyb master: Don't include retired projects for codesearch https://review.openstack.org/498919 | 19:39 |
| *** ijw has quit IRC | 19:39 | |
| Zara | (I'd initially assumed that all guides were automated, but now I'm less sure) | 19:39 |
| mordred | clarkb, fungi ^^ patch | 19:39 |
| mordred | Zara: actually, all of the guides are hand-written | 19:39 |
| mordred | we do not use anything like swagger or similar | 19:40 |
| mordred | there is no machine-parseable description of our apis | 19:40 |
| fungi | i thought there was generated api documentation as of a cycle or two ago? | 19:40 |
| mordred | this is mostly for hysterical raisins | 19:40 |
| fungi | what am i confusing it with? | 19:40 |
| *** gixxer1k has quit IRC | 19:40 | |
| fungi | maybe i'm thinking of the configuration docs | 19:40 |
| mordred | fungi: we definitely generate and publish api docs - we just generate those docs from hand-written source files | 19:40 |
| mordred | fungi: yah - those we have things to auto-generate | 19:40 |
| Zara | hm, iirc storyboard gets its docs from docstrings in the source (though those are hand-written) | 19:41 |
| fungi | for some reason i thought the api docs were at least partly scraped from docstrings | 19:41 |
| fungi | yeah, that | 19:41 |
| Zara | ( https://docs.openstack.org/infra/storyboard/webapi/v1.html ) | 19:41 |
| Zara | whereas another one I saw for a different project looked like it had been written directly into the repo | 19:42 |
| Zara | https://git.openstack.org/cgit/openstack-infra/storyboard/tree/doc/source/webapi/v1.rst | 19:43 |
| mordred | http://git.openstack.org/cgit/openstack/nova/tree/api-ref/source | 19:43 |
| mordred | that's ^^ where nova's api source code is | 19:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Create upload-afs role https://review.openstack.org/498588 | 19:43 |
| *** calebb has quit IRC | 19:43 | |
| mordred | each service project has an api-ref sphinx dir inside of which it keeps its api docs | 19:43 |
| *** calebb has joined #openstack-infra | 19:44 | |
| Zara | ah yeah, the one I saw was like that | 19:44 |
| Zara | so pretty different to storyboard | 19:44 |
| *** e0ne has quit IRC | 19:47 | |
| *** xyang1 has joined #openstack-infra | 19:47 | |
| *** dprince has joined #openstack-infra | 19:51 | |
| clarkb | dmsimard: I just quickly checked and I can ping the subnode from primary node with latest change | 19:52 |
| clarkb | dmsimard: I need to grab lunch then can do more verification | 19:52 |
| dmsimard | clarkb: good news | 19:52 |
| dmsimard | clarkb: are you holding these as well ? | 19:52 |
| clarkb | dmsimard: just the one this time | 19:53 |
| clarkb | but ya | 19:53 |
| *** e0ne has joined #openstack-infra | 19:53 | |
| dmsimard | clarkb: ok | 19:53 |
| *** jamesmcarthur has quit IRC | 19:56 | |
| pabelanger | clarkb: do you know we copy the cache around in devstack-gate? https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n533 instead of just using the cache where we create it in DIB? | 19:57 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/reviewstats master: WIP/DNM: Testing https://review.openstack.org/498878 | 19:57 |
| pabelanger | clarkb: just looking at an infracloud-chocolate job, we spend a lot time, IO moving that data around I think | 19:57 |
| clarkb | ya because rax ephemeral disk | 19:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix typo in ServerLoggingConfig https://review.openstack.org/498922 | 19:58 |
| fungi | moving it within the same device/filesystem should require basically no i/o | 19:58 |
| pabelanger | okay, so maybe it is something else | 19:58 |
| fungi | granted the way we're doing the move ends up being a lot of forks so it may not be as performant as it could be | 19:59 |
| fungi | depending on how many files/subdirectories we have there | 20:00 |
| fungi | pabelanger: https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n525 is a much more likely culprit | 20:01 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Fix a backwards boolean comparison for nodaemon logging https://review.openstack.org/498923 | 20:01 |
| fungi | since that's a copy operation and not a move | 20:01 |
| dmsimard | fungi: yay, clickable python http://logs.openstack.org/04/498904/1/check/gate-ara-integration-py35-latest-ubuntu-xenial/7fb0cc2/logs/build/reports/0bce7d46-a77e-4cd4-a4d4-894effe538aa.html | 20:01 |
| fungi | dmsimard: i feel like a real developer now--i have a commit in ara! | 20:02 |
| dmsimard | them stackalytics stats | 20:02 |
| fungi | also thanks for the pointer on where to patch that. the inconsistency was triggering my ocd tendencies ;) | 20:03 |
| openstackgerrit | Slawek Kaplonski proposed openstack-infra/shade master: Add handling timeout in servers cleanup function https://review.openstack.org/498925 | 20:03 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/reviewstats master: Update requirements.txt / test-requirements.txt https://review.openstack.org/498878 | 20:05 |
| dmsimard | np, feel free to let me know if there's anything else | 20:05 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/reviewstats master: Update requirements.txt / test-requirements.txt https://review.openstack.org/498878 | 20:06 |
| *** jamesmcarthur has joined #openstack-infra | 20:06 | |
| *** Goneri has joined #openstack-infra | 20:08 | |
| *** e0ne has quit IRC | 20:14 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Set better defaults for server logging https://review.openstack.org/498928 | 20:16 |
| *** felipemonteiro has quit IRC | 20:16 | |
| *** felipemonteiro has joined #openstack-infra | 20:16 | |
| *** felipemonteiro_ has joined #openstack-infra | 20:17 | |
| *** calebb has quit IRC | 20:18 | |
| *** kgiusti has left #openstack-infra | 20:20 | |
| *** felipemonteiro has quit IRC | 20:21 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Raise default logging level to debug if nodaemon is passed https://review.openstack.org/498931 | 20:22 |
| *** squid has joined #openstack-infra | 20:22 | |
| *** squid has quit IRC | 20:22 | |
| *** priteau has quit IRC | 20:24 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Remove default root handler fallback to console https://review.openstack.org/498635 | 20:25 |
| *** squid has joined #openstack-infra | 20:25 | |
| *** annegentle has quit IRC | 20:26 | |
| *** jkilpatr has quit IRC | 20:28 | |
| *** ijw has joined #openstack-infra | 20:28 | |
| mordred | fungi: re: nodes: [] - yup! now- there are *very* few things thatyou can usably do with a job with no nodes since we block you from executing arbitrary local code | 20:28 |
| clarkb | pabelanger: fungi the reason for it is on rax / is small so in order to have room for things like /var/log etc we move/copy the cache off of / and onto /opt which is on the ephemeral device | 20:29 |
| mordred | fungi: but we have a carve-out in our blocking of things you can run for the uri module - since jobs that ping uris without needing a vm for it are nice (such as that one) | 20:29 |
| *** annegentle has joined #openstack-infra | 20:30 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add pre-release pipeline definition to v3 config https://review.openstack.org/498907 | 20:30 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add oaktree to v3 project list for readthedocs testing https://review.openstack.org/498899 | 20:30 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add bindep_command and bindep_fallback to site-variables https://review.openstack.org/498914 | 20:30 |
| clarkb | dmsimard: yup I can ping in both directions and have confirmed that we use .1 then .3 rather than .2 | 20:31 |
| clarkb | dmsimard: a followup to fix the .2 vs .3 thing might be nice but overall this looks good | 20:31 |
| dmsimard | clarkb: would probably need to do inline jinja for the fix | 20:31 |
| dmsimard | clarkb: i.e, if index = 0, offset + 1 + index, else offset + index | 20:32 |
| mordred | clarkb, dmsimard: woot it works? | 20:32 |
| clarkb | dmsimard: also the paramiko unpinning change failed in an odd way but unrelated to paramiko from what I can tell | 20:33 |
| pabelanger | clarkb: Ya, makes sense. I'll look at infracloud-chocolate logs in a while to see why it was slow, likely something else | 20:33 |
| mordred | fungi: fixed https://review.openstack.org/#/c/498907/ - thanks! | 20:33 |
| dmsimard | mordred: progress on overlay, yes | 20:33 |
| dmsimard | clarkb: still failed tho http://logs.openstack.org/33/435933/49/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/40caa69/ | 20:33 |
| dmsimard | clarkb: ohhhh but tempest is all green this time http://logs.openstack.org/33/435933/49/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/40caa69/logs/old/testr_results.html.gz | 20:34 |
| clarkb | 2017-08-29 04:10:37.098 | fatal: ambiguous argument 'remotes/origin/master': unknown revision or path not in the working tree. | 20:34 |
| clarkb | dmsimard: cool I think that means overlay is working and the failure must be related to neutron upgrading | 20:34 |
| clarkb | I'm going to recheck the paramiko stack | 20:34 |
| dmsimard | clarkb: doh, already recheck'd it | 20:35 |
| clarkb | oh even better | 20:35 |
| mordred | clarkb: the remotes/origin/master error - where's that? | 20:35 |
| clarkb | mordred: http://logs.openstack.org/55/496355/1/gate/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/a753d40/logs/devstack-gate-setup-workspace-new.txt there | 20:35 |
| clarkb | mordred: you have to grep for it as I can't direct link | 20:35 |
| *** kjackal has quit IRC | 20:37 | |
| mordred | clarkb: wow | 20:38 |
| mordred | clarkb: lke -there are zero reasons for that patch to cause that | 20:38 |
| clarkb | ya it managed to ssh to that node according to ansible | 20:39 |
| clarkb | so paramiko should be completely unrelated | 20:39 |
| clarkb | (it doesn't even use paramiko) | 20:39 |
| clarkb | pabelanger: fungi: I've been queried to tell foundation about plans for the summit update session. They would like to know who will be speaking and if there will be more than one of us. And if we have a preferred day. Were either of you interested in joining me for the update? I'm guessing fungi is going to bask in his post ptl lack of summit updating :) | 20:41 |
| *** armax has joined #openstack-infra | 20:42 | |
| jeblair | mordred: earlier you suggested updating prepare-workspace to understand a git cache. i'd be okay with that, but openstack's git cache is not in the directory layout i would want to use for something in zuul-jobs. i propose we go with an openstack-local fork of prepare-workspace for now, and if we decide to keep the caches long-term, migrate them to golang layout and then add that to zuul-jobs. | 20:43 |
| pabelanger | clarkb: fungi: I don't mind helping speak if needed | 20:43 |
| clarkb | jeblair: we can probably use symlinks fairly easily to put them in the needed format today (then write them to the proper location once on zuulv3) | 20:45 |
| mordred | jeblair: I'm good eitherway - I mostly meant that doing it at that level of our stack seemed good, and that I could imagine it being a useful opt-in for other people | 20:46 |
| jeblair | mordred: cool | 20:46 |
| mordred | jeblair: for now, whichever thing is expedient | 20:46 |
| clarkb | dmsimard: mordred I have +2'd the overlay change | 20:46 |
| jeblair | clarkb: agree -- but i think i'd still prefer to defer anything with more than one step until post ptg :) | 20:47 |
| *** jkilpatr has joined #openstack-infra | 20:47 | |
| dmsimard | itshappening.gif | 20:47 |
| clarkb | jeblair: wfm | 20:47 |
| mordred | clarkb: \o/ | 20:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Zuul v3: Add openstack-prepare-workspace role https://review.openstack.org/498619 | 20:48 |
| jeblair | mordred, clarkb: ^ | 20:48 |
| fungi | pabelanger: i think i was the initial contact (as the ptl when they started talking about updates for sydney), i told them you had volunteered to help out, and then clarkb is a contact on it as the new ptl now | 20:48 |
| clarkb | dmsimard: mordred I think the failed pings are likely unrelated to the overlay and instead related to upgrade things. Since I separately confirmed the nodes are reachable via the overlay | 20:49 |
| fungi | clarkb: oh, wait, you were the one to say something about it a few minutes ago, i misread which nick that was | 20:49 |
| dmsimard | clarkb, mordred: Right, I'm not exactly worried about the setup of the overlay itself since before the grenade upgrade, we have a 100% pass on tempest | 20:49 |
| clarkb | dmsimard: ++ | 20:49 |
| dmsimard | before the last patchset, we had two failures related to VM networking in tempest | 20:49 |
| fungi | clarkb: i can certainly go on stage and talk about stuff. happy to help put together bits worth saying as well | 20:49 |
| clarkb | fungi: pabelanger thanks. Do either of you have a preferred day? | 20:50 |
| dmsimard | clarkb, mordred: the inventory patch needs to land before the overlay patch https://review.openstack.org/#/c/498207/8 | 20:50 |
| dmsimard | missing a +3 | 20:51 |
| mordred | yup just +3d it | 20:51 |
| pabelanger | clarkb: fungi: after tuesday? | 20:51 |
| mordred | jeblair: that's very nice | 20:52 |
| clarkb | pabelanger: conf days are monday, tuesday, wednesday | 20:52 |
| mordred | jeblair: a billion times more elegant than what I had in my head | 20:52 |
| clarkb | wednesday works for me | 20:52 |
| clarkb | jeblair: slightly confused at why there are two tasks doing copies there | 20:53 |
| clarkb | jeblair: but also would git clone be preferable to rsync so that hardlinks can be used if possible? | 20:53 |
| fungi | clarkb: no preference from me | 20:53 |
| pabelanger | clarkb: Oh, thought this was PTG :) Give me a sec | 20:53 |
| clarkb | jeblair: oh I see second one is copying over merger stage | 20:53 |
| clarkb | *state | 20:53 |
| fungi | pabelanger: sydney. you have plenty of time to freak out about it between now and then | 20:54 |
| *** annegentle has quit IRC | 20:54 | |
| clarkb | jeblair: I don't think that rsync will necessarily work that way since objects can be packed differently on different servers due to gc and repack? | 20:54 |
| fungi | that's, like, two months away | 20:54 |
| pabelanger | clarkb: fungi: Ya, wednesday is clear for me. otherwise, early day mon, or tuesday | 20:54 |
| clarkb | pabelanger: early tuesday or any time tuesday? | 20:54 |
| pabelanger | clarkb: before 3:30pm monday / tuesday, or anytime wednesday | 20:55 |
| clarkb | pabelanger: roger | 20:55 |
| *** srobert_ has joined #openstack-infra | 20:56 | |
| clarkb | jeblair: I left all that in a comment on the change | 20:57 |
| *** annegentle has joined #openstack-infra | 20:57 | |
| mordred | clarkb: I think the rsync part should work fine- there's a bunch of rsync-based tools for managing git repo state out there | 20:58 |
| clarkb | mordred: are you sure? my understanding is that this is why git archive exists | 20:59 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add trigger-readthedocs job https://review.openstack.org/498626 | 20:59 |
| clarkb | mordred: the problem is if I've recently GC'd/repacked I can have a single pack file with all objects in it but you may not | 20:59 |
| clarkb | mordred: so when you rsync you will get zero speedup | 20:59 |
| clarkb | the repo will function but your data throughput is no ebtter than just copying the entire repo | 20:59 |
| *** srobert has quit IRC | 20:59 | |
| *** florianf has quit IRC | 20:59 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix badly named function typo https://review.openstack.org/498877 | 21:00 |
| *** srobert_ has quit IRC | 21:01 | |
| *** trown is now known as trown|outtypewww | 21:02 | |
| *** yamamoto has quit IRC | 21:04 | |
| mordred | clarkb: yah - but if we're getting the repos from the same place in both cases and not doing local gc/repack, it should be fairly uncommon to hit that case, no? it'll be intersting to see how that test patch goes | 21:04 |
| *** slaweq has quit IRC | 21:04 | |
| *** ijw has quit IRC | 21:05 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul feature/zuulv3: Pin Ansible to <2.4 https://review.openstack.org/498941 | 21:05 |
| clarkb | mordred: but they are different places. Image comes from dib cache which is aggressively repacked to keep build times low. Zuul will be on its own schedule | 21:05 |
| mordred | clarkb: oh - also - there is no zuul git server for the node to fetch refs from | 21:05 |
| mordred | so we're going to have to push to the node in some form no matter what | 21:05 |
| *** gouthamr has quit IRC | 21:05 | |
| clarkb | ah ok so do a git push rather than remote update | 21:05 |
| mordred | (responding to your comment on the change) | 21:05 |
| clarkb | should still be doable | 21:06 |
| mordred | oh right - we do repack the dib cache don't we | 21:06 |
| mordred | clarkb: while I'm bothering you, could you poke https://review.openstack.org/#/c/498907/ for me? | 21:07 |
| clarkb | ya we found the build times dramatically increased by repacking I'm guessing because stat is slow | 21:07 |
| *** hasharDinner is now known as hashar | 21:07 | |
| clarkb | er dramatically shortened by repacking | 21:08 |
| clarkb | they are much logner without it | 21:08 |
| *** mat128 has quit IRC | 21:08 | |
| *** yamamoto has joined #openstack-infra | 21:08 | |
| *** rhallisey has quit IRC | 21:12 | |
| jeblair | clarkb, mordred: i did not realize the git caches were repacked on their own, so i expected the rsync to be reasonable as long as our images didn't age out more than a week. we can do this with git operations. i expect it to take me the rest of the week if we need to do it that way. | 21:13 |
| *** ldnunes has quit IRC | 21:14 | |
| clarkb | 0 20 * * * find /opt/dib_cache/source-repositories/ -type d -name "*.git" -exec git --git-dir="{}" gc \; >/dev/null | 21:15 |
| clarkb | is what we run on the nodepool builders | 21:15 |
| clarkb | in nodepool user's cron | 21:15 |
| jeblair | essentially, we'd need to do a force-push on every local ref, and a delete on any remote ref that isn't local. all from ansible. | 21:16 |
| jeblair | (ie, no use of GitPython for this) | 21:16 |
| openstackgerrit | Merged openstack-infra/project-config master: Add pre-release pipeline definition to v3 config https://review.openstack.org/498907 | 21:17 |
| jeblair | we may be able to simplify that as a force-push for every local ref, then an rsync. | 21:17 |
| *** jcoufal has quit IRC | 21:18 | |
| jeblair | clarkb, mordred: would you like me to take that approach? | 21:19 |
| clarkb | jeblair: my concern is the rsync will end up taking an hour afterall because it will copy the vast majority of each repo (due to mismatches in git objects) | 21:19 |
| clarkb | jeblair: when doing the force push it won't make the objects match up on disk it will just see they are there and push the delta. Then the rsync would copy all the files | 21:20 |
| jeblair | good point | 21:20 |
| jeblair | this is going to be a lot of ansible | 21:20 |
| jeblair | (since that leaves us with force-push local followed by delete remote refs that are not local) | 21:20 |
| *** jamesmcarthur has quit IRC | 21:21 | |
| clarkb | jeblair: you should be able to test that theory relatively easily on a held nodepool node | 21:21 |
| *** thorst_afk has quit IRC | 21:22 | |
| *** thorst_afk has joined #openstack-infra | 21:23 | |
| jeblair | clarkb: yeah, just by inspection i see that nova in the git cache has 2 packfiles | 21:25 |
| *** gixxer1k1k has quit IRC | 21:26 | |
| *** thorst_afk has quit IRC | 21:27 | |
| *** yamamoto has quit IRC | 21:30 | |
| *** yamamoto has joined #openstack-infra | 21:32 | |
| jeblair | clarkb: does this pseudocode look right? https://etherpad.openstack.org/p/sW5j0GXn2S | 21:32 |
| *** yamamoto has quit IRC | 21:32 | |
| clarkb | jeblair: yes I think so | 21:34 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Pin Ansible to <2.4 https://review.openstack.org/498941 | 21:34 |
| jeblair | dmsimard: are you interested in implementing https://etherpad.openstack.org/p/sW5j0GXn2S in ansible? | 21:35 |
| mordred | jeblair: yes - pseudocode lgtm | 21:36 |
| *** skelso has quit IRC | 21:37 | |
| *** dprince has quit IRC | 21:38 | |
| *** jascott1 has quit IRC | 21:38 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/elastic-recheck master: Add query for bug 1713832 https://review.openstack.org/498954 | 21:39 |
| openstack | bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Undecided,New] https://launchpad.net/bugs/1713832 | 21:39 |
| *** jascott1 has joined #openstack-infra | 21:39 | |
| *** ijw has joined #openstack-infra | 21:40 | |
| jeblair | mordred: oh i like --all | 21:41 |
| jeblair | mordred: oh --mirror is nice too | 21:41 |
| mordred | jeblair: I think --mirror might actually do the thing we want- except I don't believe it's going to update the working dir - so we still need to figure thatout | 21:41 |
| *** jamesmcarthur has joined #openstack-infra | 21:41 | |
| *** olaph1 has joined #openstack-infra | 21:42 | |
| jeblair | mordred: oh good point, i left that off of my initial pseudo code too | 21:42 |
| jeblair | mordred: maby we just do a 'git status' and 'git checkout' ? | 21:42 |
| *** jascott1 has quit IRC | 21:42 | |
| *** skelso has joined #openstack-infra | 21:42 | |
| *** jascott1 has joined #openstack-infra | 21:42 | |
| *** olaph has quit IRC | 21:42 | |
| jeblair | dmsimard: i think this ansible just got easy enough for even me to do :) | 21:43 |
| jeblair | i'll give this a shot | 21:43 |
| *** esberglu has quit IRC | 21:43 | |
| *** jascott1 has quit IRC | 21:43 | |
| *** esberglu has joined #openstack-infra | 21:44 | |
| *** jascott1 has joined #openstack-infra | 21:44 | |
| *** slaweq has joined #openstack-infra | 21:44 | |
| jeblair | mordred: (also, maybe we want to think about splitting zuul_console from prepare_workspace?) | 21:44 |
| *** annegentle has quit IRC | 21:45 | |
| mordred | jeblair: ++ | 21:45 |
| *** tbarron is now known as tbarron|errand | 21:46 | |
| *** jascott1 has quit IRC | 21:46 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Improve function to find PR from commit status https://review.openstack.org/498957 | 21:46 |
| *** jascott1 has joined #openstack-infra | 21:47 | |
| jeblair | i can with_items a block, right? | 21:48 |
| *** esberglu has quit IRC | 21:48 | |
| *** jamesmcarthur has quit IRC | 21:48 | |
| bkero | Sure, it's just expecting a yaml list | 21:48 |
| *** jascott1 has quit IRC | 21:49 | |
| *** slaweq has quit IRC | 21:49 | |
| *** thorst_afk has joined #openstack-infra | 21:49 | |
| bkero | Sorry, yaml calls them sequences | 21:49 |
| jeblair | apparently not? https://github.com/ansible/ansible/issues/13262 | 21:49 |
| mordred | jeblair: butyah - I think git status and git checkout | 21:49 |
| bkero | Oh, thought you meant a yaml block >_> | 21:50 |
| mordred | jeblair: and no - I think you need to do the with items on each thing - so shell: git push --mirror with_items: repos ; shell: git checkout {{ branch }} with_items: repos | 21:50 |
| jeblair | mordred: hrm, that makes the 'git status' + 'git checkout' weird | 21:51 |
| *** ijw has quit IRC | 21:51 | |
| jeblair | i need to run 'git status' on localhost and then 'git checkout' on remote for each project | 21:51 |
| mordred | jeblair: yah - SO - there's a fun thing you could do ... | 21:52 |
| mordred | jeblair: http://docs.ansible.com/ansible/latest/playbooks_loops.html#looping-over-parallel-sets-of-data | 21:53 |
| *** thorst_afk has quit IRC | 21:53 | |
| *** felipemonteiro_ has quit IRC | 21:53 | |
| jeblair | mordred: not seeing the applicability :( | 21:53 |
| mordred | jeblair: if you do git status: with_items: repos register: statuses ... then I think you could do git checkout with_together: - repos - statuses | 21:53 |
| jeblair | oh, hrm | 21:54 |
| mordred | since the first loop will make a list of status results | 21:54 |
| jeblair | yes, i think that fits, thanks | 21:54 |
| mordred | jeblair: we may have found the one use case for that command | 21:54 |
| jeblair | hehe | 21:55 |
| pabelanger | clarkb: fungi: http://logs.openstack.org/20/497720/1/gate/gate-tripleo-ci-centos-7-scenario004-multinode-oooq-puppet/315538d/logs/devstack-gate-setup-workspace-new.txt on infracloud-vanilla. that failed to setup workspace, and job timed out | 21:56 |
| *** esberglu has joined #openstack-infra | 21:57 | |
| *** marst has joined #openstack-infra | 21:58 | |
| *** jascott1 has joined #openstack-infra | 22:00 | |
| *** baoli has quit IRC | 22:01 | |
| *** marst_ has quit IRC | 22:01 | |
| *** baoli has joined #openstack-infra | 22:01 | |
| *** esberglu has quit IRC | 22:01 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: WIP https://review.openstack.org/498102 | 22:04 |
| mordred | pabelanger, clarkb: https://review.openstack.org/#/c/498899 and https://review.openstack.org/#/c/498914 if you get a sec? | 22:04 |
| clarkb | pabelanger: 2017-08-29 21:18:26.401 | error: RPC failed; result=35, HTTP code = 0 when cloning gnocchi | 22:04 |
| *** sdague has quit IRC | 22:04 | |
| dmsimard | mordred, jeblair: I had another example use case for with_together in the ansibilization of the inventory if you want to look :p https://review.openstack.org/#/c/498207/6/playbooks/setup_inventory.yaml@49 | 22:04 |
| clarkb | do we even host a working gnocchi anymore? | 22:05 |
| *** olaph1 is now known as olaph | 22:05 | |
| dmsimard | it's not that arcane | 22:05 |
| mordred | clarkb: was that cloning from github? | 22:05 |
| *** xyang1 has quit IRC | 22:06 | |
| mordred | clarkb: and no, the gnocchi in git.o.o is empty with a readme | 22:06 |
| mordred | dmsimard: hey lookie there! | 22:07 |
| jeblair | well, we *do* host a gnocchi. it's empty. but it is what devstack-gate uses. | 22:07 |
| *** esberglu has joined #openstack-infra | 22:07 | |
| clarkb | Are we doing a proper clone because it isn't in our cache? | 22:08 |
| dmsimard | jeblair: seems like you've already started hacking on the git stuff, let me know if you have anything else in mind. Otherwise I'll continue to help where I can | 22:08 |
| jeblair | clarkb: probably | 22:08 |
| clarkb | and even if the current working contents are empty that clone may be non trivial depending on hsitory | 22:08 |
| jeblair | oh yeah, it's not small | 22:09 |
| mordred | I don't see gnocchi referenced in d-g - is that because it's just in a PROJECTS list? | 22:09 |
| clarkb | mordred: ya I think so | 22:09 |
| *** hashar has quit IRC | 22:09 | |
| *** Goneri has quit IRC | 22:10 | |
| mordred | {trigger}-tripleo-ci-{jobname}{suffix} in jenkins/jobs/tripleo.yaml adds gnocchi | 22:10 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Remove gnocchi from PROJECTS for tripleo-ci https://review.openstack.org/498965 | 22:11 |
| mordred | pabelanger, clarkb: ^^ | 22:11 |
| *** bobh has quit IRC | 22:11 | |
| clarkb | mordred: does tripleo install it from there though? they may actually need it | 22:12 |
| clarkb | EmilienM: ^ | 22:12 |
| EmilienM | clarkb: currently dealing with tripleo ci fires right no | 22:12 |
| jeblair | clarkb: if that's passing tests then i don't think the tests are good | 22:12 |
| mordred | EmilienM: yah - that's actually in response to one of your test failures | 22:12 |
| clarkb | EmilienM: us too :P | 22:12 |
| jeblair | git status |head -1 |awk '{ print $NF }' | 22:13 |
| jeblair | mordred, clarkb: ^ | 22:13 |
| EmilienM | pabelanger: any change to promote 498772 to the gate now please? | 22:13 |
| mordred | jeblair: woot | 22:13 |
| EmilienM | mordred, clarkb: ok, let me read context then | 22:13 |
| EmilienM | clarkb: from where? | 22:13 |
| *** Goneri has joined #openstack-infra | 22:13 | |
| mordred | jeblair: git rev-parse --abbrev-ref HEAD | 22:13 |
| jeblair | mordred: fails on detached head (eg a tag checkout) | 22:14 |
| jeblair | (which we would have in a release job) | 22:14 |
| mordred | jeblair: ah! fascinating | 22:14 |
| clarkb | EmilienM: https://review.openstack.org/498965 basically that change. tripleoci is cloning gnocchi which is not cached by infra because it is a deprecated repo. It is large and fails to clone at times | 22:14 |
| *** dtroyer has quit IRC | 22:14 | |
| EmilienM | clarkb: could we please promote 498772 to the gate now ? | 22:14 |
| clarkb | EmilienM: so wondering if it is safe to just stop cloning it in tripleoci jobs | 22:14 |
| EmilienM | clarkb: ok let me look | 22:15 |
| EmilienM | clarkb: we don't install anything from source | 22:15 |
| EmilienM | clarkb: but use RDO packages | 22:15 |
| EmilienM | so +1 | 22:15 |
| clarkb | EmilienM: they why are we cloning anything? | 22:15 |
| clarkb | pabelanger: ^ | 22:15 |
| EmilienM | I don't know, tbh | 22:15 |
| mordred | jeblair: yes - I fully agree that your thing is the most accurate | 22:16 |
| clarkb | and re 498772 that was promoted once already right? Are we sure it actually fixes the problem before promoting again? | 22:16 |
| EmilienM | clarkb: yes | 22:16 |
| clarkb | jeblair: mordred sorry I feel like i am lacking context | 22:16 |
| clarkb | re awk | 22:16 |
| EmilienM | clarkb: it fix one of our 3 problems | 22:16 |
| clarkb | EmilienM: ok | 22:17 |
| EmilienM | it's very appreciated | 22:17 |
| jeblair | clarkb: last stage of syncing is to checkout on the remote node what was checked out on the executor | 22:17 |
| mordred | clarkb: "get the current branch of tag that is checked out in the current local repository" | 22:17 |
| jeblair | clarkb: approach is 'git status' on local then 'git checkout' on remote | 22:17 |
| * mordred tried to come up with fancier git commands to one-up jeblair but failed | 22:17 | |
| clarkb | mordred: jeblair ah, keep in mind that you can have same tag on multiple branches potentially | 22:17 |
| clarkb | (so you might need additional filtering to pick one?) | 22:17 |
| * jeblair read like 200 pages of stackoverflow before doing the awk thing (which he did not actually see in stackoverflow) | 22:17 | |
| clarkb | EmilienM: I will promote it once this mistral change merges | 22:18 |
| clarkb | since it is close | 22:18 |
| *** bobh has joined #openstack-infra | 22:18 | |
| EmilienM | clarkb: wfm | 22:18 |
| jeblair | clarkb: there should only be one thing checked out though, and git status tells us what it is. some approaches (eg rev-parse) fail in the way you describe, but i don't think this does. | 22:18 |
| clarkb | also why are networking-bgp and mistral in the tripleo gate? | 22:18 |
| *** ijw has joined #openstack-infra | 22:18 | |
| clarkb | jeblair: gotcha | 22:18 |
| EmilienM | clarkb: the only thing I know is that these 2 projects run tripleo jobs in gate | 22:18 |
| *** Goneri has quit IRC | 22:20 | |
| *** lbragstad has quit IRC | 22:20 | |
| clarkb | EmilienM: that will do it but I don't see those jobs running currently | 22:21 |
| mordred | jeblair: yup. awk wins here | 22:21 |
| EmilienM | clarkb: see https://review.openstack.org/#/c/494749/ | 22:22 |
| EmilienM | clarkb: gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet is running | 22:22 |
| *** bobh has quit IRC | 22:22 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add mirror-workspace-git-repos role https://review.openstack.org/498967 | 22:23 |
| jeblair | mordred, dmsimard: ^ how's that look? | 22:23 |
| *** ijw has quit IRC | 22:23 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Add the private_ipv4 hostvar to primary and sub nodes https://review.openstack.org/498207 | 22:23 |
| *** Apoorva has joined #openstack-infra | 22:24 | |
| pabelanger | clarkb: so, just catching up | 22:24 |
| *** mwarad has joined #openstack-infra | 22:24 | |
| *** annegentle has joined #openstack-infra | 22:25 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add start-zuul-console role https://review.openstack.org/498968 | 22:26 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade master: Support domain_id for user operations https://review.openstack.org/497483 | 22:26 |
| EmilienM | clarkb: 498772 can be promoted now | 22:26 |
| *** esberglu has quit IRC | 22:27 | |
| *** esberglu has joined #openstack-infra | 22:27 | |
| clarkb | EmilienM: yup in progress | 22:27 |
| pabelanger | clarkb: dlrn it still cloning from github.com, but they have a patch to install from pip. I am not sure about gnocchi ATM | 22:27 |
| clarkb | pabelanger: EmilienM says they don't install anything from git so we should maybe not clone any projects | 22:28 |
| *** esberglu_ has joined #openstack-infra | 22:28 | |
| EmilienM | clarkb: thanks | 22:28 |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Move network overlay configuration to Ansible playbooks https://review.openstack.org/435933 | 22:28 |
| EmilienM | pabelanger: gnocchi is deployed as other packages | 22:29 |
| jeblair | dmsimard, rcarrillocruz, clarkb: \o/ ^ thanks! | 22:29 |
| EmilienM | like ceilometer for ex | 22:29 |
| pabelanger | clarkb: right, I need to see what they do for DLRN again, I think they use the git repos for rebuilding rpms | 22:29 |
| *** esberglu has quit IRC | 22:31 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Improve function to find PR from commit status https://review.openstack.org/498957 | 22:32 |
| *** baoli has quit IRC | 22:33 | |
| *** ijw has joined #openstack-infra | 22:34 | |
| *** esberglu_ has quit IRC | 22:35 | |
| openstackgerrit | Kendall Nelson proposed openstack-infra/storyboard master: Adds Migration Documentation https://review.openstack.org/496439 | 22:38 |
| *** wolverineav has quit IRC | 22:38 | |
| *** gouthamr has joined #openstack-infra | 22:38 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Move fix_disk_layout to an Ansible role in setup_host https://review.openstack.org/496935 | 22:38 |
| *** ijw has quit IRC | 22:38 | |
| *** wolverineav has joined #openstack-infra | 22:38 | |
| *** rlandy is now known as rlandy|bbl | 22:38 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove gnocchi from PROJECTS for tripleo-ci https://review.openstack.org/498965 | 22:38 |
| *** dtroyer has joined #openstack-infra | 22:40 | |
| *** Apoorva has quit IRC | 22:42 | |
| *** Apoorva has joined #openstack-infra | 22:42 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/elastic-recheck master: Remove query for bug 1674681 https://review.openstack.org/498974 | 22:43 |
| openstack | bug 1674681 in tripleo "buildlogs.centos.org CDN issues" [Critical,In progress] https://launchpad.net/bugs/1674681 | 22:43 |
| *** wolverineav has quit IRC | 22:43 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for bug 1713832 https://review.openstack.org/498954 | 22:44 |
| openstack | bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] https://launchpad.net/bugs/1713832 - Assigned to Emilien Macchi (emilienm) | 22:44 |
| mordred | jeblair: comments left - looks good, one is just musing, the other is, I believe, an actual problem | 22:45 |
| *** ntpttr has joined #openstack-infra | 22:45 | |
| ntpttr | hey all, I was wondering how to make a depends-on cross project issue work | 22:46 |
| jlvillal | Does "Depends-On:" work for unit test type jobs? 'python-jobs' | 22:46 |
| ntpttr | the patch in question: https://review.openstack.org/#/c/496410/ | 22:46 |
| ntpttr | it's failing due to not finding a class that was added in the patch that it depends on | 22:46 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade master: Add domain_id to groups https://review.openstack.org/497537 | 22:46 |
| *** Apoorva has quit IRC | 22:47 | |
| clarkb | generally unittest type jobs don't install their deps from source so no | 22:47 |
| clarkb | jlvillal: ^ | 22:47 |
| jlvillal | clarkb: Thanks. I was thinking that was the case. | 22:47 |
| ntpttr | gotcha | 22:47 |
| clarkb | requirements is an exception to this rule | 22:47 |
| clarkb | but ya | 22:47 |
| jlvillal | ntpttr: You could maybe do a requirements.txt change to make it work? For testing purposes. Maybe | 22:47 |
| ntpttr | guess I've got to wait for the patch to merge, which I would've had to do to merge it anyways, so not a huge deal | 22:48 |
| jlvillal | ntpttr: I'm not quite sure how to do it... | 22:48 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Zuul v3: Add use-cached-repos role https://review.openstack.org/498619 | 22:48 |
| jeblair | mordred: looking | 22:48 |
| ntpttr | probably not worth the effort | 22:48 |
| jeblair | jlvillal, clarkb, ntpttr: mordred has an idea to make that very easy to do in zuul v3; hopefully we should be able to do that soon | 22:49 |
| jlvillal | jeblair: Cool :) | 22:49 |
| ntpttr | awesome | 22:49 |
| ntpttr | thanks jeblair | 22:50 |
| ntpttr | and mordred | 22:50 |
| *** skelso has quit IRC | 22:50 | |
| jeblair | jlvillal, ntpttr: https://review.openstack.org/489719 in case you're curious | 22:51 |
| ntpttr | I'll check it out | 22:51 |
| EmilienM | https://review.openstack.org/435933 tripleo CI | 22:51 |
| EmilienM | jeblair: ^ | 22:51 |
| EmilienM | +broke | 22:51 |
| jeblair | EmilienM: got a link to a failure log? | 22:52 |
| pabelanger | sounds like path issue or package didn't get installed properly | 22:52 |
| pabelanger | jeblair: http://logs.openstack.org/72/498772/2/gate/gate-tripleo-ci-centos-7-undercloud-oooq/a8af258/logs/undercloud/home/jenkins/vxlan_networking.sh.log.txt.gz | 22:52 |
| EmilienM | pabelanger: with the number of issues that we have in the pipe, I would likely avoid to do a patch in tripleo to fix the vxlan stuff | 22:52 |
| EmilienM | pabelanger: because the patch will probably fail because of our 3 other issues | 22:52 |
| EmilienM | 2017-08-29 22:43:24 | /home/jenkins/vxlan_networking.sh: line 91: ovs_vxlan_bridge: command not found | 22:53 |
| EmilienM | I'm not sure we should do such changes during a release | 22:54 |
| jeblair | that was the name of a function that was removed | 22:54 |
| mordred | jlvillal, ntpttr : I've got it half-written - it's just on the backburner for the leadup to v3 rollout - will be totally doable | 22:54 |
| pabelanger | Ya, looks like tripleo-ci is calling that function directly | 22:54 |
| mordred | s/doable/easy/ once I get one little bit written | 22:54 |
| jlvillal | mordred: cool. will be useful for the case ntpttr is doing :) | 22:55 |
| mordred | jlvillal: ++ | 22:55 |
| clarkb | fungi: pabelanger https://etherpad.openstack.org/p/infra-sydney-update how does that look for sydney update description? | 22:55 |
| EmilienM | pabelanger, jeblair I'm proposing a revert | 22:55 |
| jeblair | pabelanger, EmilienM: i'd suggest copying that function into tripleo-ci | 22:55 |
| openstackgerrit | Akihiro Motoki proposed openstack-infra/project-config master: OSC: Allow feature branch merge commits in gerrit https://review.openstack.org/498981 | 22:55 |
| EmilienM | and I'm happy to change tripleo to stop using this function once our gate is back | 22:56 |
| jeblair | pabelanger, EmilienM: devstack-gate isn't a general purpose shell function library | 22:56 |
| clarkb | or call the ansible playbook? | 22:56 |
| EmilienM | jeblair: like I said, our gate is really broken at this time | 22:56 |
| EmilienM | jeblair: ok sorry I didn't write that code in our CI | 22:56 |
| EmilienM | jeblair: i can't tell anything but sorry | 22:56 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul-jobs master: Add a role to remove an ssh private key https://review.openstack.org/498530 | 22:57 |
| *** mwarad has quit IRC | 22:57 | |
| jeblair | EmilienM: i don't think you need to apologize :) | 22:57 |
| EmilienM | so we have 2 options : | 22:57 |
| clarkb | we can also shim the function back in such that it calls the ansible playbook | 22:57 |
| EmilienM | 1) revert https://review.openstack.org/#/c/435933/ | 22:57 |
| clarkb | though I guess that assumes you've installed ansible which may not be the case? | 22:57 |
| EmilienM | 2) fix tripleo and pray the patch won't fail on our 3 other problems in gate presently | 22:57 |
| pabelanger | jeblair: EmilienM: So, tripleo-ci should likely move devstack-gate/functions.sh in-tree, they seem to be depending on a few things. And zuulv3 effort is going to break them is some good ways | 22:58 |
| EmilienM | jeblair: I'll git blame to see who decided to use devstack-gate code but it was a terrible idea | 22:59 |
| EmilienM | jeblair: since d-g doesn't gate on tripleo jobs | 22:59 |
| mordred | WELL - for the most part our v3 plan is 2 parallel thing - so there actually isn't much plan to modify more things | 22:59 |
| clarkb | EmilienM: I think the suggestion was to use devstack gate as devstack gate | 22:59 |
| clarkb | EmilienM: but it sounds like instead you are just calling the function direclty | 22:59 |
| *** rbrndt has quit IRC | 22:59 | |
| EmilienM | clarkb: which is bad, I figured | 22:59 |
| mordred | so other than this, I don't expect we'll be doing much more with touching devstack-gate functions or modifying them | 22:59 |
| clarkb | EmilienM: there is a flag that you can flip in d-g to have it make an overlay for you which would be the right way to do it | 22:59 |
| clarkb | rather than invoking it directly | 22:59 |
| pabelanger | clarkb: ya, for multinode jobs they are using devstack-gate functions to setup overlay themself | 23:00 |
| pabelanger | so, I agree with jeblair. They likely need to in-tree that function and work towards migrating to zuulv3 at / after PTG | 23:00 |
| jeblair | pabelanger: yes, i think that's the best way forward. we need to keep moving on zuulv3 (so that we actually *do* have things designed to be used by tripleoci -- things are going to get a lot better when we do). that's probably the best interim solution. | 23:00 |
| EmilienM | bkero wrote that code (which use the d-g function) | 23:01 |
| pabelanger | Ya, otherwise, I'm going to get pulled into fixing tripleo-ci but downstream :) | 23:01 |
| pabelanger | let me get a patch for tripleo-ci | 23:01 |
| * bkero ears perk up | 23:02 | |
| *** annegentle has quit IRC | 23:02 | |
| EmilienM | https://github.com/openstack/tripleo-quickstart-extras/commit/d4c4290355d12ebab2869d0b02d26a9590ee7621 | 23:02 |
| EmilienM | pabelanger: thanks | 23:02 |
| *** ijw has joined #openstack-infra | 23:02 | |
| EmilienM | pabelanger: I just hope the patch won't fail on some other problems we're solving | 23:02 |
| bkero | EmilienM: AFAIK it's always been sourced from d-g. I just ansible-ified it. | 23:03 |
| EmilienM | bkero: indeed | 23:03 |
| pabelanger | EmilienM: well, possible something else will break as we roll to PTG and zuulv3. We'll have to keep an eye and have to help keep things running | 23:03 |
| bkero | (It was d-g before the commit referenced) | 23:03 |
| EmilienM | bkero: I'm trying to understand if we do that elsewhere | 23:03 |
| mordred | well - so ... | 23:04 |
| bkero | EmilienM: I hope not. | 23:04 |
| mordred | bkero: since you're calling the shell function from ansible- perhaps instead you could just call the ansible? or is that too much surgery for now? | 23:04 |
| jeblair | mordred: that has the same problem -- d-g isn't meant to be a general purpose ansible library either :) | 23:05 |
| bkero | mordred: I'm not working on that project anymore. It should certainly be possible though. | 23:05 |
| mordred | jeblair: indeed. yah - I thnk copy-pasta into local repo is lowest overhead thing | 23:05 |
| jeblair | mordred: (at least, not yet) | 23:05 |
| bkero | mordred: slagle -2'd when I tried that | 23:06 |
| bkero | code duplication something something | 23:06 |
| jeblair | copy-pasta to fix and make us safe, then rebuild the jobs as zuulv3 native multinode jobs would be great | 23:06 |
| mordred | yah | 23:07 |
| mordred | bkero: totes. and I think made sense then - for now though I think it's a good bandaid until the new multinode jobs can be used | 23:07 |
| bkero | mordred: good luck, and let me know if you have any questions about the implementation and how to transition. | 23:08 |
| *** ijw has quit IRC | 23:09 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add mirror-workspace-git-repos role https://review.openstack.org/498967 | 23:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add start-zuul-console role https://review.openstack.org/498968 | 23:11 |
| jeblair | mordred: ^ fixed | 23:11 |
| mordred | jeblair: I believe you has a space in the remote git url | 23:13 |
| mordred | between host and user_dir | 23:13 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade master: Allow domain_id for roles https://review.openstack.org/496992 | 23:13 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci master: In-tree ovs_vxlan_bridge function from devstack-gate https://review.openstack.org/498985 | 23:13 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add mirror-workspace-git-repos role https://review.openstack.org/498967 | 23:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add start-zuul-console role https://review.openstack.org/498968 | 23:14 |
| jeblair | mordred: ^ fixed :) | 23:14 |
| mordred | yay! | 23:14 |
| mordred | jeblair: one more thing - in the first section, it's item.src_dir not item.0.src_dir | 23:15 |
| *** tosky has quit IRC | 23:15 | |
| mordred | jeblair: but this is fun! | 23:15 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add mirror-workspace-git-repos role https://review.openstack.org/498967 | 23:16 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add start-zuul-console role https://review.openstack.org/498968 | 23:16 |
| *** baoli has joined #openstack-infra | 23:16 | |
| jeblair | mordred: ^ fixed :) | 23:16 |
| mordred | jeblair: boomshakalaka | 23:16 |
| *** ianychoi has quit IRC | 23:17 | |
| pabelanger | jeblair: mordred: dmsimard: swap issues? http://logs.openstack.org/85/498985/1/check/gate-tripleo-ci-centos-7-undercloud-containers-nv/1cc3b99/logs/devstack-gate-setup-host.txt | 23:17 |
| pabelanger | /dev/xvde1: No such file or directory | 23:17 |
| mordred | mayhap clarkb or pabelanger or jlk will +A you there https://review.openstack.org/#/c/498967 | 23:17 |
| *** ianychoi has joined #openstack-infra | 23:17 | |
| *** ijw has joined #openstack-infra | 23:17 | |
| pabelanger | 2017-08-29 23:15:41.598 | localhost | FAILED | fix_disk_layout : Fix the disk layout | rc=1 >> | 23:19 |
| clarkb | pabelanger: yes that is tracked by elastic-recheck | 23:19 |
| clarkb | the sync and partprobe there were an attempt at fixing it | 23:19 |
| pabelanger | clarkb: k, thanks. Just unlucky I guess | 23:19 |
| clarkb | but unfortunately that isn't enough to ensure linux knowsabout the new partition before running mkswap | 23:19 |
| pabelanger | clarkb: thanks, I see the elastic-recheck query now | 23:20 |
| pabelanger | just bad luck on that patch | 23:20 |
| *** stakeda has joined #openstack-infra | 23:22 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade master: Add domain_id to groups https://review.openstack.org/497537 | 23:24 |
| pabelanger | mordred: +3 | 23:24 |
| pabelanger | oh, jlk also did | 23:24 |
| pabelanger | :) | 23:24 |
| *** marst has quit IRC | 23:24 | |
| *** baoli has quit IRC | 23:25 | |
| *** annegentle has joined #openstack-infra | 23:25 | |
| mordred | pabelanger: \o/ | 23:25 |
| mordred | pabelanger: you could +3 https://review.openstack.org/#/c/498899/ and https://review.openstack.org/#/c/498914/2 if you wanted to ;) | 23:26 |
| * mordred hands pabelanger a nice plate of pie | 23:27 | |
| pabelanger | looking | 23:27 |
| jlk | it's log pie | 23:27 |
| pabelanger | done and done | 23:28 |
| *** yamamoto has joined #openstack-infra | 23:28 | |
| jlk | or you know, 1.144729885849400...... | 23:28 |
| jlk | log(Pi) | 23:28 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade master: Support domain_id for user operations https://review.openstack.org/497483 | 23:28 |
| dmsimard | pabelanger, clarkb: wait, so is there an issue or not ? | 23:28 |
| jlk | (I'll see myself out) | 23:28 |
| dmsimard | From looking at http://logs.openstack.org/85/498985/1/check/gate-tripleo-ci-centos-7-undercloud-containers-nv/1cc3b99/logs/ara/result/95b57eb5-8899-45a5-bf17-444ed65550d3/ | 23:29 |
| clarkb | dmsimard: yes tripleo ci will need to do something to fix it. Either vendor the function or file or use the ansible role directly | 23:29 |
| dmsimard | clarkb: would a "sync" after the partprobe help ? | 23:30 |
| dmsimard | ah I see the partprobe and the sync have already been added for that | 23:31 |
| clarkb | dmsimard: I don't think so. partprobe is what is supposed to load it into /dev | 23:31 |
| dmsimard | does it just happen too fast ? | 23:31 |
| clarkb | and sync is a bad attempt at makingsure the partition is on disk (sync operates on filesystems though which may be the problem) | 23:31 |
| clarkb | dmsimard: that is the assumption since it works most of the time | 23:31 |
| clarkb | so assumption is we are racing and devfs isn't updated yet | 23:31 |
| dmsimard | what's the bugid for elastic recheck ? | 23:32 |
| jeblair | mordred: did you lookt at https://review.openstack.org/498619 ? | 23:32 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade master: Support domain_id for user operations https://review.openstack.org/497483 | 23:32 |
| clarkb | dmsimard: http://status.openstack.org/elastic-recheck/index.html#1706103 | 23:32 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Improve function to find PR from commit status https://review.openstack.org/498957 | 23:33 |
| dmsimard | 3 fails in 10 days, eh | 23:33 |
| *** spzala has joined #openstack-infra | 23:33 | |
| *** spzala has quit IRC | 23:33 | |
| *** claudiub|2 has quit IRC | 23:33 | |
| clarkb | ya it only happens in rax (because that is the only place we make a swap partition | 23:33 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade master: Support domain_id for user operations https://review.openstack.org/497483 | 23:33 |
| *** yamahata has quit IRC | 23:34 | |
| mordred | jeblair: mostly awesome - one gotcha in there though | 23:35 |
| openstackgerrit | Merged openstack-infra/project-config master: Add oaktree to v3 project list for readthedocs testing https://review.openstack.org/498899 | 23:35 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix a backwards boolean comparison for nodaemon logging https://review.openstack.org/498923 | 23:36 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Set better defaults for server logging https://review.openstack.org/498928 | 23:37 |
| *** rossella_s has quit IRC | 23:37 | |
| *** rossella_s has joined #openstack-infra | 23:39 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Raise default logging level to debug if nodaemon is passed https://review.openstack.org/498931 | 23:39 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove default root handler fallback to console https://review.openstack.org/498635 | 23:40 |
| *** yamahata has joined #openstack-infra | 23:40 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Create upload-afs role https://review.openstack.org/498588 | 23:41 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add mirror-workspace-git-repos role https://review.openstack.org/498967 | 23:42 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add start-zuul-console role https://review.openstack.org/498968 | 23:43 |
| *** annegentle has quit IRC | 23:43 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Zuul v3: Add use-cached-repos role https://review.openstack.org/498619 | 23:44 |
| jeblair | mordred: gah, thanks! :) | 23:44 |
| *** yamamoto has quit IRC | 23:46 | |
| mordred | jeblair: oh - wait a sec ... I think I may have given you a bad review in an earlier change ... | 23:47 |
| mordred | jeblair: or - no - nevermind. however- I think before name: Clone cached repo to workspace you need to do a file: state: directory on dirname(item.0.src_dir) | 23:49 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Make a docs-on-readthedocs project-template https://review.openstack.org/498903 | 23:50 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix typo in ServerLoggingConfig https://review.openstack.org/498922 | 23:50 |
| jeblair | mordred: clone will make the target subdirs | 23:50 |
| jeblair | at arbitrary depth even | 23:50 |
| mordred | oh - it will? NEAT | 23:50 |
| mordred | jeblair: than that patch is great | 23:51 |
| *** vhosakot has quit IRC | 23:52 | |
| jeblair | i'll push it through then and rebase devstack-legacy | 23:53 |
| dmsimard | jeblair: did the git ansible module not work ? | 23:53 |
| jeblair | dmsimard: didn't try. it sounds like a lot of work to map low-level git ops to high level. :) | 23:54 |
| dmsimard | jeblair: well, just for the clones ? I dunno. | 23:55 |
| dmsimard | like, I don't think there's a way to remove a ref with the git module | 23:55 |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Unpin Paramiko https://review.openstack.org/496355 | 23:59 |
| clarkb | jeblair: does that only work for a single repo? | 23:59 |
| jeblair | clarkb: whatcha mean? | 23:59 |
| clarkb | jeblair: specifically the stat seems to only check a single item | 23:59 |
| clarkb | jeblair: what happens if zuul.projects is a list of projects (whihc I think it can be right?) | 23:59 |
| jeblair | clarkb: 498619 main.yaml line 18? | 23:59 |
| clarkb | oh derp | 23:59 |
| clarkb | the reason the others are indexed is because they are with_together | 23:59 |
| clarkb | nevermind | 23:59 |
| jeblair | clarkb: ya. projects is a list, and in later tasks, it's zipped with the results of the stats | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
