Wednesday, 2016-12-07

« Tuesday, 2016-12-06 Index Thursday, 2016-12-08 »
*** askb has quit IRC00:04
*** askb has joined #openstack-release00:07
*** claudiub has joined #openstack-release00:28
*** claudiub|2 has quit IRC00:31
*** wolverineav has joined #openstack-release00:34
*** wolverineav has quit IRC00:39
*** wolverineav has joined #openstack-release00:44
*** sdake has joined #openstack-release00:45
*** wolverineav has quit IRC00:50
openstackgerritMerged openstack-infra/release-tools: remove the old dashboard command  https://review.openstack.org/40629100:54
*** wolverineav has joined #openstack-release00:58
*** hongbin has quit IRC01:00
*** wolverineav has quit IRC01:03
*** dave-mccowan has joined #openstack-release01:04
*** Jeffrey4l has quit IRC01:06
*** Jeffrey4l has joined #openstack-release01:06
*** Jeffrey4l has quit IRC01:08
*** Jeffrey4l has joined #openstack-release01:09
*** sdake has quit IRC01:14
*** Jeffrey4l has quit IRC01:20
*** Jeffrey4l has joined #openstack-release01:21
*** wolverineav has joined #openstack-release01:21
*** wolverineav has quit IRC01:27
*** david-lyle has joined #openstack-release01:33
*** mriedem has joined #openstack-release01:36
*** claudiub has quit IRC01:37
*** Jeffrey4l has quit IRC01:39
*** Jeffrey4l has joined #openstack-release01:39
*** yanyanhu has joined #openstack-release01:48
*** sdake has joined #openstack-release02:01
*** Jeffrey4l has quit IRC02:06
*** Jeffrey4l has joined #openstack-release02:07
*** hongbin has joined #openstack-release02:46
*** sdake has quit IRC02:47
*** Jeffrey4l has quit IRC03:23
*** Jeffrey4l has joined #openstack-release03:24
*** dave-mccowan has quit IRC03:25
*** wolverineav has joined #openstack-release03:26
*** dave-mccowan has joined #openstack-release03:27
*** Jeffrey4l has quit IRC03:38
*** Jeffrey4l has joined #openstack-release03:38
*** wolverineav has quit IRC03:40
*** spzala has quit IRC03:42
*** spzala has joined #openstack-release03:42
*** spzala has quit IRC03:47
*** spzala has joined #openstack-release04:00
*** spzala has quit IRC04:00
*** dave-mccowan has quit IRC04:06
*** Jeffrey4l has quit IRC04:11
*** Jeffrey4l has joined #openstack-release04:11
*** armax has quit IRC04:21
*** Jeffrey4l has quit IRC04:29
*** Jeffrey4l has joined #openstack-release04:30
*** Jeffrey4l has quit IRC04:30
*** Jeffrey4l has joined #openstack-release04:31
*** bobh has joined #openstack-release04:43
*** Rocky_g has quit IRC04:45
*** bobh has quit IRC04:58
*** hongbin has quit IRC04:58
*** sdake has joined #openstack-release05:01
*** sdake has quit IRC05:01
*** sdake has joined #openstack-release05:01
*** jamespage has quit IRC05:02
*** jamespag` has joined #openstack-release05:02
*** sdake_ has joined #openstack-release05:04
*** sdake has quit IRC05:07
*** hyakuhei has quit IRC05:14
*** robcresswell has quit IRC05:14
*** zhenguo_ has joined #openstack-release05:14
*** claudiub has joined #openstack-release05:14
*** onovy has quit IRC05:14
*** zhenguo has quit IRC05:15
*** dirk has quit IRC05:15
*** samueldmq_ has joined #openstack-release05:15
*** zhenguo_ is now known as zhenguo05:15
*** betherly has quit IRC05:16
*** samueldmq has quit IRC05:16
*** samueldmq_ is now known as samueldmq05:17
*** robcresswell has joined #openstack-release05:17
*** betherly has joined #openstack-release05:19
*** dirk has joined #openstack-release05:19
*** onovy has joined #openstack-release05:20
*** hyakuhei has joined #openstack-release05:31
*** Jeffrey4l has quit IRC05:35
*** Jeffrey4l has joined #openstack-release05:35
*** armax has joined #openstack-release05:49
*** oanson has joined #openstack-release05:54
*** ramishra has joined #openstack-release06:13
*** armax has quit IRC06:18
*** sdake_ has quit IRC06:36
*** sdake has joined #openstack-release06:36
*** jgriffith is now known as jgriffith_away06:39
*** sdake_ has joined #openstack-release06:47
*** sdake has quit IRC06:50
*** sdake_ has quit IRC07:08
*** sileht has quit IRC07:13
*** kzaitsev_mb has joined #openstack-release07:30
*** pcaruana has joined #openstack-release07:41
*** jpich has joined #openstack-release07:44
*** shardy has joined #openstack-release07:46
*** sdake has joined #openstack-release07:55
*** mriedem has quit IRC08:01
*** ihrachys has joined #openstack-release08:05
*** Jeffrey4l has quit IRC08:17
*** Jeffrey4l has joined #openstack-release08:17
*** zul has quit IRC08:18
*** sileht has joined #openstack-release08:26
*** zul has joined #openstack-release08:32
*** Rocky_g has joined #openstack-release08:36
*** kzaitsev_mb has quit IRC08:36
*** Rocky_g has quit IRC08:38
*** Rocky_g has joined #openstack-release08:38
*** Rocky__ has joined #openstack-release08:39
*** Rocky__ has quit IRC08:39
*** Rocky_g has quit IRC08:42
*** Rocky_g has joined #openstack-release08:42
*** ramishra has left #openstack-release08:43
*** Rocky_g has quit IRC08:46
*** Rocky_g has joined #openstack-release08:47
*** Rocky_g has quit IRC08:48
*** Rockyg has quit IRC08:49
*** Rockyg has joined #openstack-release08:50
*** daemontool_ has quit IRC08:59
*** sdake has quit IRC09:03
*** csomerville has quit IRC09:08
*** csomerville has joined #openstack-release09:09
*** eandersson has joined #openstack-release09:22
*** srikanthchandika has joined #openstack-release09:27
*** eandersson has quit IRC09:28
*** odyssey4me_ is now known as odyssey4me09:30
*** lucas-afk is now known as lucasagomes09:32
*** jpich has quit IRC09:37
*** electrofelix has joined #openstack-release09:46
*** jpich has joined #openstack-release09:47
*** kzaitsev_mb has joined #openstack-release09:55
*** kzaitsev_mb has quit IRC10:00
*** kzaitsev_mb has joined #openstack-release10:02
*** dtantsur|afk is now known as dtantsur10:03
openstackgerritMerged openstack/releases: add missing info to newton deliverables  https://review.openstack.org/40713210:09
openstackgerritDirk Mueller proposed openstack/releases: New tripleo release to uncap docker-py requirements  https://review.openstack.org/40644310:16
openstackgerritMerged openstack/releases: import stable/newton branch history  https://review.openstack.org/40633210:20
*** david-lyle_ has joined #openstack-release10:35
*** david-lyle has quit IRC10:37
*** yanyanhu has quit IRC10:46
openstackgerrityolanda.robla proposed openstack/releases: Release python-tripleoclient 5.6.0  https://review.openstack.org/40699810:47
*** csomerville has quit IRC10:53
*** sdake has joined #openstack-release10:55
*** dtantsur is now known as dtantsur|brb10:56
*** kzaitsev_mb has quit IRC10:57
*** shardy has quit IRC10:58
*** kong_ has quit IRC10:58
*** kong_ has joined #openstack-release10:59
*** claudiub has quit IRC11:01
*** wolverineav has joined #openstack-release11:06
*** kzaitsev_mb has joined #openstack-release11:06
openstackgerrityolanda.robla proposed openstack/releases: Release python-tripleoclient 5.6.0  https://review.openstack.org/40699811:08
*** sdague has joined #openstack-release11:08
*** wolverineav has quit IRC11:10
ttxdhellmann: FYI merging https://review.openstack.org/#/c/406332/ seems to have triggered a lot of things, including LP comment addition -- might be worth reviewing that output: http://logs.openstack.org/70/7057ebaeff72af16c7a3fc5df7482db35a4b8602/release-post/tag-releases/5a8b881/console.html11:13
ttxdhellmann: also would like to have your re-review on https://review.openstack.org/#/c/398401 before approving11:18
openstackgerritMerged openstack/releases: Show team and repo badges on README  https://review.openstack.org/40240511:19
ttxHm. Looks like the "sudo: no tty present and no askpass program specified" error is back11:22
*** tommylikehu_ has joined #openstack-release11:22
ttxnot merging anything until we get to the bottom of it11:26
*** askb has quit IRC11:29
*** openstackgerrit has quit IRC11:32
*** hyakuhei has quit IRC11:51
*** hyakuhei has joined #openstack-release11:51
*** hyakuhei has quit IRC11:51
*** hyakuhei has joined #openstack-release11:51
*** openstackgerrit has joined #openstack-release11:54
openstackgerrityolanda.robla proposed openstack/releases: Release python-tripleoclient 5.6.0  https://review.openstack.org/40699811:54
*** dtantsur|brb is now known as dtantsur12:00
*** tommylikehu_ has quit IRC12:03
*** tommylikehu_ has joined #openstack-release12:04
*** dave-mccowan has joined #openstack-release12:09
*** rtheis has joined #openstack-release12:19
*** oanson has quit IRC12:28
*** gordc has joined #openstack-release12:45
*** lucasagomes is now known as lucas-hungry12:49
dhellmannttx: oh, I'll bet the logic around updating launchpad isn't skipped if the release already exists12:58
ttxdhellmann: do you remember what caused the sudo issue ?13:00
ttxsome change on the image we are using iirc13:00
dhellmannttx: I don't remember, but that sounds likely. Maybe we fungi remembers?13:01
fungiwe allow jobs to sudo on single-use nodes13:02
fungiwe don't allow jobs to sudo on persistent nodes13:02
ttxit's the signing node13:02
ttxhttp://logs.openstack.org/70/7057ebaeff72af16c7a3fc5df7482db35a4b8602/release-post/tag-releases/5a8b881/console.html13:03
fungilooking at the job log to see if i can figure out the underlying cause13:03
dhellmannttx, is that actually an error?13:03
dhellmannwe see that message all the time. I thought it meant that the job had correctly removed its sudo privileges13:03
ttxdhellmann: the job ends up failing, couldn't spot any other error in it13:04
ttxthere was a shorter one, stand by13:04
ttxhttp://logs.openstack.org/ff/ffb57b88079de72d775e4a2c5a0a6f26bbb10626/release-post/tag-releases/c060e66/console.html13:04
dhellmannit looks like the job took 35 minutes, what's our timeout?13:04
ttxdhellmann: the second I posted is short13:04
dhellmannah13:04
ttxnad failed too13:04
ttxand*13:04
dhellmannoh, that's weird13:05
dhellmannright after an exit 013:05
fungicould launchpad_add_comment.py be silently exiting nonzero13:05
fungi?13:05
dhellmannthat's possible, but it's not run in that second job13:05
ttxfungi: see the second one, same failure13:05
*** bobh has joined #openstack-release13:05
*** bobh has quit IRC13:05
ttxremoves a lot of noise13:06
ttxit was a doc change13:06
*** bobh has joined #openstack-release13:06
fungiyeah, launchpad_add_comment.py seems to have successfully updated the bugs in that first one anyway13:06
dhellmannthe builder step doesn't explicitly exit 013:07
dhellmannbut zuul reported the task as exiting 013:07
dhellmanncould it be something about handling the logs after the job is actually done? or is that reported elsewhere?13:08
fungithe publisher steps are logged by ansible, but very dense. i'm attempting to interpret http://logs.openstack.org/ff/ffb57b88079de72d775e4a2c5a0a6f26bbb10626/release-post/tag-releases/c060e66/_zuul_ansible/ansible_log.txt13:09
*** mriedem has joined #openstack-release13:09
dhellmannI don't see any obvious error messages there13:10
*** jpich has quit IRC13:11
fungiit looks like release_from_yaml.sh ran, but i don't see branch_from_yaml.sh mentioned in the console log13:12
fungiaccording to http://logs.openstack.org/ff/ffb57b88079de72d775e4a2c5a0a6f26bbb10626/release-post/tag-releases/c060e66/_zuul_ansible/scripts/03-16a24b26f880473ea0d6f32750d840c2.sh it would have done both13:13
dhellmannhttp://logs.openstack.org/ff/ffb57b88079de72d775e4a2c5a0a6f26bbb10626/release-post/tag-releases/c060e66/console.html#_2016-12-07_11_19_50_11272613:13
dhellmannthere are no branches, so there's very little output from it13:14
fungioops, sorry, i switched back to the other console log. that explains it13:14
sigmavirusmriedem: great, I'll propose that and I'd appreciate your review on that13:17
mriedemsigmavirus: ok13:18
*** claudiub has joined #openstack-release13:20
*** cody-somerville has joined #openstack-release13:22
*** cody-somerville has quit IRC13:22
*** cody-somerville has joined #openstack-release13:22
dhellmannttx: https://review.openstack.org/40808513:22
fungii'm currently digging in the zuul-launcher logs on zlstatic01 to see if i can make heads or tails of this13:23
dhellmannfungi : is it interesting that there is almost a full second between the "task exit code: 0" line and the "job complete" line in the log?13:23
dhellmannI don't know how long that usually takes13:23
fungithere's stuff getting logged in http://logs.openstack.org/ff/ffb57b88079de72d775e4a2c5a0a6f26bbb10626/release-post/tag-releases/c060e66/_zuul_ansible/ansible_log.txt during that missing second13:24
ttxdhellmann: +113:29
*** bobh has quit IRC13:30
dhellmannfungi : ah, sure, that makes sense13:30
funginothing in the zl log really other than what's in the ansible log for the job13:34
*** trown|outtypewww is now known as trown13:34
fungiis this a copy/paste error from a coverage job? http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/release-jobs.yaml#n2913:36
fungilooks like it was introduced with the original job definition by https://review.openstack.org/341058 so presumably not the cause13:37
*** cody-somerville has quit IRC13:40
dhellmannfungi : copy and paste error?13:43
*** myoung|bbl is now known as myoung|alt13:43
*** myoung|alt is now known as myoung13:43
fungidhellmann: wondering why the tag-releases job would generate logs under a subdirectory named "cover"13:43
dhellmannah, I see, yeah, likely a copy-pasta13:43
dhellmannfungi : I'll propose a fix for that. Is there a convention for that directory name? same as the job? just "releases"?13:45
dhellmannfungi : oh, that's the *source* of the logs13:46
fungiright, wherever that job creates extra log files you want collected besides stdout13:47
dhellmannah13:47
fungiand if none, we can just get rid of that publisher entirely and only use the console-log publisher for that job13:47
dhellmannso we don't need that block at all, because we don't write any extra logs13:47
dhellmanngot it13:48
fungiright then13:48
fungistill seems unlikely to have cause the job failure13:48
fungier, caused13:48
dhellmanntrue, but we might as well clean it up: https://review.openstack.org/40812613:49
dhellmannI have to step out, but I should be back in <1hr13:49
ttxfungi: if that helps, the error seems to have appeared during the night13:55
ttxwe merged https://review.openstack.org/#/c/407719/ without error13:55
fungiroughly when was the last successful run>?13:55
fungiaha13:55
ttx22:31 UTC13:55
*** rhallisey has joined #openstack-release13:56
ttxthis one did not trigger a release-failures email, so I assume it was successful13:56
ttxhttp://lists.openstack.org/pipermail/release-job-failures/2016-December/thread.html shows the fails started later13:56
ttx10:55 UTC being the earliest13:57
*** lucas-hungry is now known as lucasagomes13:58
*** jpich has joined #openstack-release13:58
fungiokay, so http://logs.openstack.org/7b/7bfce311c673c474112be090b881229facad45a4/release-post/tag-releases/c442ad5/ corresponds to the last known good run13:58
fungiand it does report a success result13:59
ttxand sudo failed the same, so that's probably expected set up13:59
fungiyeah, that comes from the revoke-sudo builder. this job doesn't need it because it runs on a host which lacks sudo permissions for the jenkins user anyway, but it's a no-op14:00
*** srikanthchandika has quit IRC14:00
ttxcan't find any change from last night that would affect that14:02
fungithe ansible log from the last successful run shows more tasks than the one from the first failure14:03
ttxfungi: at least the /tc/ change on governance works14:03
fungion the successful run, after the third TASK [command generated from JJB] there's a TASK [file] before it moves on to PLAY [Publishers]14:04
fungiin the failing one, it goes straight from the last TASK [command generated from JJB] to PLAY [Publishers]14:05
*** sdake_ has joined #openstack-release14:05
ttxhmm, can't spot a difference14:08
*** sdake has quit IRC14:09
fungiand that isn't the case with the second failure14:11
ttxyeah, I'm comparing second fail with last known-good, and there is no diff, except the ordering14:11
fungii'm starting to suspect that the two failed runs failed for different reasons14:11
ttxskipping: [node] => {"changed": false, "skip_reason": "Conditional check failed", "skipped": true} appears in a slightly different order, probably a parallelized check14:12
ttxeverything else is just PID / datetime changes14:12
fungithe first failure seems like it stopped suddenly. the default build-timeout wrapper defined at http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/defaults.yaml#n21 stops jobs at the 30-minute mark unless the job itself overrides that (which tag-releases does not)14:14
ttxyeah, it was probably stuck updating LP due to the other bug14:14
fungiand that job went for 35 minutes best i can see14:15
ttxi'm fine attempting another release, I have one queued up14:15
ttx(a "normal" one)14:15
fungii'm still worried about the second failure, for which i can find no explanation of the failure14:15
ttxwe could just blame flaper8714:20
flaper87ttx: sounds reasonable to me14:22
flaper87wait, what did I do?14:22
flaper87:D14:22
* flaper87 badges himself as failures-source14:23
ttxflaper87: we should add that as a principle. "When all else fails, blame flaper87"14:23
flaper87ttx: rofl14:23
*** mriedem has quit IRC14:28
ttxfungi: shall we try to run the next release ? It's not as if it actually failed, it just seems to report failure erroneously but proceed to tag the release alright14:28
*** Jeffrey4l has quit IRC14:30
*** Jeffrey4l has joined #openstack-release14:31
fungittx: yeah, we can still manually retrigger each of the dependent jobs after14:31
fungii guess try one more and let's see14:31
ttxok approving https://review.openstack.org/#/c/406443/14:33
openstackgerritMerged openstack/releases: New tripleo release to uncap docker-py requirements  https://review.openstack.org/40644314:38
ttxtag-releases job being run as I speak14:39
ttxfailure14:40
ttxhttp://logs.openstack.org/78/787bed80b04917525e1113c5e12c48130a4c2563/release-post/tag-releases/2d9b80e/14:40
ttxmain issue being it skips the publish job14:41
ttxthe other jobs are triggered alright14:42
*** trown has quit IRC14:48
openstackgerritDavanum Srinivas (dims) proposed openstack/releases: Add links to the package signatures  https://review.openstack.org/40719014:48
*** trown has joined #openstack-release14:50
ttxfungi: apparently tarball-signing jobs also inexplicably fail14:52
ttxhttp://logs.openstack.org/b7/b7f1fdb032eebfa95ee16c06602e5472da54f7b3/release/tripleo-common-tarball-signing/87f8b14/console.html14:52
*** jamespag` is now known as jamespage14:53
*** mriedem has joined #openstack-release14:55
fungiyeah, and those run on the same static node as the tag jobs, maintained by the same launcher14:57
ttxto be more exact, they report failure, rather than fail14:57
*** hongbin has joined #openstack-release15:07
*** armax has joined #openstack-release15:14
*** wolverineav has joined #openstack-release15:23
*** daemontool has joined #openstack-release15:23
*** dave-mccowan has quit IRC15:23
*** Jeffrey4l has quit IRC15:27
*** Jeffrey4l has joined #openstack-release15:27
*** jpich has quit IRC15:28
*** sdake_ has quit IRC15:29
*** jpich has joined #openstack-release15:29
fungiyep. i've got mordred and clarkb involved now too, similarly confused by this15:29
mordredit's WEIRD15:29
mordredflaper87: congratulations - the rule used to be "When all else fails, blame jaypipes"15:29
fungiwell, there was a time where we just blamed vish15:30
flaper87ok, now I feel honored. I never thought I'd share such an important title with folks like jay and vish15:31
* flaper87 drops a tear and bows proudly15:31
*** sdake has joined #openstack-release15:32
*** dave-mccowan has joined #openstack-release15:41
*** spzala has joined #openstack-release15:46
*** cody-somerville has joined #openstack-release15:57
*** cody-somerville has quit IRC15:57
*** cody-somerville has joined #openstack-release15:57
*** wolverineav has quit IRC15:58
*** wolverineav has joined #openstack-release15:58
dimsdhellmann : we haven't found anyone to update the lazr.restful on pypi?16:05
*** wolverineav has quit IRC16:06
dhellmannttx, fungi : sorry, errands took longer than expected this morning; catching up now16:08
dhellmanndims : I haven't looked in a while. Maybe see about zul?16:09
*** jgriffith_away is now known as jgriffith16:09
dimszul : coreycb : looks like xnox (https://launchpad.net/~xnox) has at least touched lazr.restfulclient (http://bazaar.launchpad.net/~lazr-developers/lazr.restfulclient/trunk/files). Would you be able to ping him about updating pypi? (https://bugs.launchpad.net/lazr.authentication/+bug/1609128)16:09
openstackLaunchpad bug 1609128 in lazr.authentication "PyPI metadata is wrong" [Undecided,New]16:09
dimsack dhellmann16:09
dimswas digging up our friends :)16:09
zuldims: hes on #ubuntu-devel16:10
dimszul : cool will show up there16:10
*** armax has quit IRC16:12
*** armax has joined #openstack-release16:14
*** ihrachys has quit IRC16:15
*** armax has quit IRC16:17
*** jpich has quit IRC16:17
*** pcaruana has quit IRC16:18
dimsdhellmann zul : pinged a couple of folks there, cross my fingers16:18
dhellmanndims : ack16:19
*** spzala has quit IRC16:19
*** armax has joined #openstack-release16:19
dhellmanndims : I think because of the immutable nature of pypi, the fix will be a 0.1.4 release with the proper metadata16:20
dimsdhellmann : yep understood16:20
*** spzala has joined #openstack-release16:20
*** spzala has quit IRC16:23
*** armax has quit IRC16:25
*** wolverineav has joined #openstack-release16:29
*** spzala has joined #openstack-release16:31
*** wolverineav has quit IRC16:34
*** spzala has quit IRC16:35
*** armax has joined #openstack-release16:37
*** spzala has joined #openstack-release16:37
*** spzala has quit IRC16:41
*** kzaitsev_mb has quit IRC16:43
*** spzala has joined #openstack-release16:48
*** wolverineav has joined #openstack-release16:50
*** armax has quit IRC16:51
*** Kiall has quit IRC16:53
*** spzala has quit IRC16:53
*** Kiall has joined #openstack-release16:53
*** spzala has joined #openstack-release16:54
*** wolverineav has quit IRC16:54
*** spzala has quit IRC16:59
*** tommylikehu_ has quit IRC17:01
*** kzaitsev_mb has joined #openstack-release17:01
*** bobh has joined #openstack-release17:05
*** spzala has joined #openstack-release17:06
*** kzaitsev_mb has quit IRC17:06
*** wolverineav has joined #openstack-release17:10
*** spzala has quit IRC17:11
ttxdhellmann: we retried one and the tag job reported failure again (triggering the skip of the publish job). tarball-signing jobs also report failure (while completing successfully)17:11
*** spzala has joined #openstack-release17:12
*** bobh has quit IRC17:12
*** wolverineav has quit IRC17:14
*** spzala has quit IRC17:16
*** dtantsur is now known as dtantsur|afk17:19
*** kzaitsev_mb has joined #openstack-release17:21
*** ihrachys has joined #openstack-release17:22
*** spzala has joined #openstack-release17:22
*** kzaitsev_mb has quit IRC17:25
*** spzala has quit IRC17:27
*** rhallisey has quit IRC17:27
*** rhallisey has joined #openstack-release17:28
*** spzala has joined #openstack-release17:28
*** wolverineav has joined #openstack-release17:30
*** spzala has quit IRC17:33
*** dhellmann has quit IRC17:33
*** wolverineav has quit IRC17:35
*** lucasagomes is now known as lucas-afk17:35
*** daemontool has quit IRC17:35
*** spzala has joined #openstack-release17:40
*** spzala has quit IRC17:44
*** spzala has joined #openstack-release17:46
*** spzala has quit IRC17:50
*** wolverineav has joined #openstack-release17:50
*** electrofelix has quit IRC17:55
*** wolverineav has quit IRC17:55
*** trown is now known as trown|lunch17:55
*** dhellmann has joined #openstack-release18:00
*** wolverineav has joined #openstack-release18:01
*** david-lyle_ is now known as david-lyle18:02
*** wolverineav has quit IRC18:05
*** spzala has joined #openstack-release18:06
dhellmannttx: mordred has a fix for zuul https://review.openstack.org/#/c/408194/118:08
dhellmannor really https://review.openstack.org/#/c/408194 since there have been updates18:08
*** bobh has joined #openstack-release18:09
*** spzala has quit IRC18:12
*** spzala has joined #openstack-release18:12
*** dhellmann has quit IRC18:15
*** bobh has quit IRC18:15
*** dhellmann has joined #openstack-release18:15
*** cody-somerville has quit IRC18:17
*** spzala has quit IRC18:17
*** cody-somerville has joined #openstack-release18:19
*** cody-somerville has quit IRC18:19
*** cody-somerville has joined #openstack-release18:19
*** spzala has joined #openstack-release18:23
*** spzala has quit IRC18:27
fungidhellmann: ttx: not sure if you followed the explanation there, but basically the first tag job failure which i was discounting as unrelated because it seemed to have been caused by a timeout was _actually_ the problem and caused all subsequent jobs run on that node to get treated as timeout-related failures (because the launcher didn't correctly reset a necessary timer between jobs)18:33
dhellmannfungi : I'm glad we were able to help find this edge case. ;-)18:34
fungithis was a crazy one to track down. kudos to pabelanger and mordred18:34
dhellmannindeed, thanks pabelanger & mordred18:35
fungistill fiddling with the fix now to get it to pass zuul's unit tests18:35
fungiso not out of the woods just yet18:35
dhellmannwhen that merges, will it be deployed automatically, or does that involve releasing and then restarting services?18:35
fungiwe'll need to restart zuul launchers to pick up the change (i think) but that's fast in the case of the one where we're hitting this (zlstatic01)18:36
dhellmannk18:36
fungii mean, restarting the launcher now would get those jobs to stop failing, but we run the risk that another lp api timeout gets us right back into this state again18:36
fungiso probably best to avoid exercising it until we have the fix deployed18:37
*** wolverineav has joined #openstack-release18:41
*** spzala has joined #openstack-release18:43
*** wolverineav has quit IRC18:45
*** spzala_ has joined #openstack-release18:45
*** armax has joined #openstack-release18:46
*** spzala_ has quit IRC18:47
*** spzala_ has joined #openstack-release18:47
*** spzala has quit IRC18:48
mriedemsigmavirus: is asking for a unit test too much here? https://review.openstack.org/#/c/408196/19:00
mriedemoops19:00
* mriedem goes to -stable19:00
*** wolverineav has joined #openstack-release19:01
*** ihrachys has quit IRC19:01
*** armax has quit IRC19:03
*** wolverineav has quit IRC19:06
*** armax has joined #openstack-release19:06
*** bobh has joined #openstack-release19:11
*** topol has left #openstack-release19:13
*** bobh has quit IRC19:15
*** wolverineav has joined #openstack-release19:21
*** kzaitsev_mb has joined #openstack-release19:22
*** wolverineav has quit IRC19:26
*** ianychoi has quit IRC19:30
*** cody-somerville has quit IRC19:31
*** wolverineav has joined #openstack-release19:32
*** ihrachys has joined #openstack-release19:35
*** wolverineav has quit IRC19:36
*** armax has quit IRC19:37
*** trown|lunch is now known as trown19:42
*** ihrachys has quit IRC19:49
*** dhellmann_ has joined #openstack-release19:52
*** ihrachys has joined #openstack-release19:55
*** ihrachys has quit IRC19:55
*** ihrachys has joined #openstack-release19:56
*** ihrachys has quit IRC19:56
*** ihrachys has joined #openstack-release19:57
fungidhellmann: ttx: oh, and i restarted zlstatic01 with that timeout fix about an hour ago and everything seems safe to proceed. can someone put together a list of the jobs that got skipped and i'll work on retriggering those manually?19:58
*** dhellmann has quit IRC19:58
*** dhellmann_ is now known as dhellmann19:59
*** gordc has quit IRC20:00
*** ihrachys has quit IRC20:02
dhellmannfungi : ack, thanks, let me see if I can pull that list together20:04
dhellmannfungi : see the notes in https://etherpad.openstack.org/p/cdgeRIqLKD20:10
dhellmannit looks like only one important job failed, the tripleo-common-tarball-signing job on line 920:10
dhellmannthe tag-releases failure on line 20 was for where we added the branch history for newton, so there shouldn't have been any work to do and that job does NOT need to rerun20:11
*** bobh has joined #openstack-release20:11
dhellmannthe publish job that was skipped will be re-run when we approve the next release, so you don't need to worry about that one either20:12
dhellmannthere were some other failures for unofficial projects, and I can pull that info together, but I would wait for the PTL to request a rerun20:13
fungidhellmann: the tarball signing job didn't break though, it merely erroneously reported as failed but still generated and uploaded signatures right?20:14
* fungi double checks that they uploaded20:14
dhellmannfungi : ok, if that's the case then I think there's nothing to rerun20:15
fungidhellmann: ahh, nope i'm wrong. the publisher for the signatures only runs on success of the job builders20:15
fungii'll retrigger it now20:15
fungiunlike log publishers which we configure to run even when a builder fails20:15
*** spzala_ has quit IRC20:16
fungi(this is intended to guard against the possibility of accidentally uploading bad signature data or something)20:16
*** bobh has quit IRC20:16
dhellmannmakes sense20:16
*** spzala has joined #openstack-release20:17
*** spzala has quit IRC20:21
fungidhellmann: tripleo-common-5.6.0.tar.gz.asc and tripleo_common-5.6.0-py2-none-any.whl.asc show up at https://tarballs.openstack.org/tripleo-common/ now so should be all set20:23
fungiwere there no release announcement jobs that this caused to get skipped?20:23
dhellmannfungi : great, thanks20:23
dhellmannapparently not20:23
dhellmannline 1220:23
fungioh, right, since we don't upload signatures to pypi we opted to not make the pypi upload job depend on the signing job20:24
fungiand since zuul can't make one job depend on the success of more than one other job (strict tree dependency hierarchy), we announce the release even if the signing job fails20:24
sdakedhellmann quick question - Jeffrey4 (proxy ptl for Kolla while inc0 is schelpping his posessions across the country) has delegated an item for me to answer relating to process of the kolla-kubernetess deliverable20:25
sdakedhellmann and I need your help - since you know this stuff inside and out :)20:25
sdakekolla-kubernetes is release-independent20:25
sdakeI saw in some ptl mail somewhere in the past that release-independent deliverables are expected to follow the release schedule published for ocata20:26
dhellmanncycle-with-intermediary are, but not independent20:26
dhellmannwe freeze independent releases at the same time, but we don't expect them to follow the schedule20:26
dhellmannthe cycle is the thing they're "independent of"20:27
sdakeinc0 has given direction that we plan to release 1.0.0 in ocata20:27
sdakeok, so i guess its a misunderstanding on my part20:27
sdakebut i had read soemwhere in the gov docs that for a release to be part of the integrated release, it must hit two milestones on schedule20:28
dhellmannyes, but by definition independent projects are not part of the release20:28
dhellmannagain, that's what they're independent of20:28
sdakei see20:28
dhellmannif you want to release at any time, but be part of the release, use cycle-with-intermediary20:28
sdakeso we can tag 1.0.0 whenever we like and not run afoul of the process?20:28
sdakewell we are stuck with release independent at the moemnt ;)20:29
dhellmannwhy's that?20:29
sdakei thought there was a deadline tot change release models?20:29
dhellmannah, yeah, true20:30
sdakeagain - delegated, not really my responsibility however jeffrey4l has asked em to get clarification20:30
dhellmannok, so, yes, you can release when you're ready20:30
sdakeok, i thought we were locked into releasing 1.0.0.0b2 on the 15th of december20:30
sdakewhich is a big commitment20:31
sdakeit indicates we will be ready for 1.0.0 in ocata20:31
sdakeso the right answer here is we can tag 1.0.0.0b2 when its ready20:31
sdakeand tag 1.0.0.0b3 when its ready20:32
sdakeand tag 1.0.0 when its ready20:32
sdakeis that an accurate assessment?20:32
dhellmannwell, independent projects don't really use pre-release versioning because that makes it look like they're milestone-based projects20:32
dhellmannso you'd want to pick a version less than 1.0.0 that isn't a pre-release20:32
dhellmannyou're at 0.3.0 now, so you could just go 0.4.0, 0.5.0, etc.20:32
sdakeand then tag 1.0.0 when we re ready?20:33
dhellmannand then when you're ready for 1.0.0, tag that one20:33
dhellmannright20:33
sdakeand there isn't a specific or implied requirement we follow the schedule precisely?20:33
sdakefor tht particular deliverlable20:34
dhellmannthat's correct. you can release at any time that we do not have a general release freeze in place20:34
sdakecool thanks a bunch for the clarification20:34
sdakei'll change the traciker aroudn because we have been operating as a milestone release cycle deliverable with kolla-kubernets20:35
sdaketracker being launchpad20:35
dhellmannk20:36
sdakei guess the only issue is inc0 expects kolla-kubernetes to release with kolla-ansible and kolla deliverables for their final release20:36
sdakenot my problem to solve; if you have any insight on how to solve that would appreciate it20:37
sdakei'm going to link this log to the core reviewer team so they grasp the full context20:37
sdakeas well as ourproxy-ptl and ptl :)20:37
dhellmannthat's just a timing thing, so it shouldn't be an issue20:38
sdakewith the change to the ptg structure and summit events, i think people are a little unesy with the relesae model and runup to summit for demo time20:39
sdakeright a timing thing20:39
dhellmannthere ought to be plenty of time between the ptg and summit to demo ocata at the summit20:39
sdakeagreed20:40
sdakei suspect people arn't aaware of that ;)20:40
sdakewhich spawned this whole conversation in the first place ;)20:40
*** harlowja has quit IRC20:41
sdakethanks again as always a pleasure20:41
dhellmannany time :-)20:41
*** wolverineav has joined #openstack-release20:42
*** TravT has joined #openstack-release20:42
*** wolverineav has quit IRC20:47
*** rtheis has quit IRC20:48
*** spzala has joined #openstack-release20:51
*** wolverineav has joined #openstack-release20:52
*** spzala has quit IRC20:55
*** wolverineav has quit IRC20:57
*** spzala has joined #openstack-release21:01
*** spzala has quit IRC21:03
*** spzala has joined #openstack-release21:03
*** askb has joined #openstack-release21:07
*** spzala has quit IRC21:07
*** wolverineav has joined #openstack-release21:13
*** cody-somerville has joined #openstack-release21:15
*** cody-somerville has quit IRC21:15
*** cody-somerville has joined #openstack-release21:15
*** csomerville has joined #openstack-release21:16
*** wolverineav has quit IRC21:17
*** cody-somerville has quit IRC21:20
*** kzaitsev_mb has quit IRC21:29
*** Jeffrey4l has quit IRC21:35
*** Jeffrey4l has joined #openstack-release21:35
*** harlowja has joined #openstack-release21:41
*** trown is now known as trown|outtypewww21:58
*** rhallisey has quit IRC21:59
*** csomerville has quit IRC22:02
*** mriedem has quit IRC22:06
*** csomerville has joined #openstack-release22:06
*** dave-mccowan has quit IRC22:09
*** bobh has joined #openstack-release22:13
*** bobh has quit IRC22:18
*** wolverineav has joined #openstack-release22:24
*** wolverin_ has joined #openstack-release22:27
*** spzala has joined #openstack-release22:29
*** wolverineav has quit IRC22:29
*** armax has joined #openstack-release22:33
*** spzala has quit IRC22:36
*** sdake has quit IRC22:37
*** wolverin_ has quit IRC22:38
openstackgerritIsaku Yamahata proposed openstack/releases: networking-odl: Release networking-odl newton/3.1.1  https://review.openstack.org/40833522:38
*** ianychoi has joined #openstack-release22:40
*** mriedem has joined #openstack-release22:55
*** armax has quit IRC23:03
*** wolverineav has joined #openstack-release23:08
*** armax has joined #openstack-release23:12
*** tommylikehu_ has joined #openstack-release23:14
*** wolverineav has quit IRC23:15
*** harlowja has quit IRC23:16
*** sdake has joined #openstack-release23:16
*** david-lyle_ has joined #openstack-release23:20
*** david-lyle_ has quit IRC23:20
*** sdake has quit IRC23:20
*** tommylikehu_ has quit IRC23:23
*** armax has quit IRC23:34
*** csomerville has quit IRC23:38
*** trown|outtypewww has quit IRC23:40
*** trown has joined #openstack-release23:42
*** sdake has joined #openstack-release23:50
*** sdake has quit IRC23:57
*** sdake has joined #openstack-release23:58
« Tuesday, 2016-12-06 Index Thursday, 2016-12-08 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!