Monday, 2014-12-22

*** e0ne has quit IRC00:02
fungialexpilotti: i have a confirmed working fix, i'm just checking it now for regressions00:05
fungii'll release it as 0.10.7 sometime tonight hopefully, reviews willing00:06
alexpilottifungi: cool tx, I can also test it if you could please send me the review link00:06
jd__pbr /o\00:14
jd__thanks fungi :)00:14
*** mtanino has joined #openstack-infra00:16
*** fandi has quit IRC00:17
*** Masahiro has joined #openstack-infra00:18
*** hdd has joined #openstack-infra00:22
*** Masahiro has quit IRC00:22
fungialexpilotti: https://review.openstack.org/14333700:25
alexpilottifungi: thanks!00:27
*** yolanda has quit IRC00:29
*** hichtakk has joined #openstack-infra00:32
alexpilottifungi: worked perfectly fine, tried without the tag and by forcing the version in setup.cfg as it used to be00:32
*** hdd has quit IRC00:36
*** hichtakk has quit IRC00:41
*** carl_baldwin has joined #openstack-infra00:41
*** salv-orlando has joined #openstack-infra00:41
*** alexpilotti has quit IRC00:43
*** salv-orlando has quit IRC00:46
*** avozza is now known as zz_avozza00:47
*** grue_pm has quit IRC00:49
*** otter768 has joined #openstack-infra00:50
*** grue_pm has joined #openstack-infra00:51
*** otter768 has quit IRC00:55
jheskethfungi: so with the zuul cloner on slaves we're doing a git clone of zuul and installing it to a venv.. With the swift push stuff, it's not its own project. So should we A) fetch the swift uploader .py file from the gitweb, or B) Split the swift uploader into its own project00:57
*** carl_baldwin has quit IRC00:58
*** Masahiro has joined #openstack-infra01:01
*** teran has joined #openstack-infra01:02
*** teran has quit IRC01:03
*** signed8bit has joined #openstack-infra01:03
*** teran has joined #openstack-infra01:04
*** teran_ has quit IRC01:04
*** teran_ has joined #openstack-infra01:06
*** teran__ has joined #openstack-infra01:07
fungijhesketh: i think we can just make a virtualenv and then call whatevenv/bin/python thatscript.py01:07
fungijhesketh: we can continue obtaining the script from wherever we've been getting it in those jobs so far01:08
*** teran has quit IRC01:08
*** FlorianSW|away has joined #openstack-infra01:08
jheskethfungi: the script is done as a slave-script from a builder01:08
jheskethso I'd have to change all the nodes to call it from within the virtualenv01:08
jheskethwhich is probably okay01:09
*** teran_ has quit IRC01:10
*** teran__ has quit IRC01:11
FlorianSW|awayhello all! Maybe someone can help with a little problem? Actually i want to migrate from gerrit trigger (jenkins plugin) to zuul. I have setup zuul and a first check pipeline to cover a test project in gerrit (should start a build when comment "recheck" is added). I also installed the gearman plugin (build from source) in jenkins and added a test job ("jenkins-test" which i have added to the check pipeline in the test01:12
FlorianSW|awayand see (in zuul.log and debug.log), that zuul added it to gearman, but jenkins doesn't start the job. In jenkins log file i don't see anything :/ Can someone assist me?01:12
*** FlorianSW|away is now known as FlorianSW01:12
*** asettle is now known as asettle-BRB01:12
*** lttrl has quit IRC01:13
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Set up venv for zuul-swift-logs  https://review.openstack.org/14232701:18
openstackgerritJoshua Hesketh proposed openstack-infra/system-config: Remove python magic  https://review.openstack.org/14233001:18
jheskethfungi: ^01:19
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Set up venv for zuul-swift-logs  https://review.openstack.org/14232701:20
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Start using the zuul-swift-upload venv  https://review.openstack.org/14334001:20
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add support for 'connection' concept  https://review.openstack.org/12152801:26
*** yolanda has joined #openstack-infra01:31
*** carl_baldwin has joined #openstack-infra01:31
*** jerryz has joined #openstack-infra01:34
*** unicell has joined #openstack-infra01:35
*** stevemar has joined #openstack-infra01:39
FlorianSWto my above question: hmm, it seems, that zuul sends a SUBMIT_JOB_LOW and gets JOB_CREATED and a NOOP, then sends a GRAB_JOB_UNIQ and receives a NO_JOB. This seems to be not correct, because the created job should be there? :/01:41
*** Ryan_Lane has quit IRC01:46
*** ryanpetrello has joined #openstack-infra01:46
*** Ryan_Lane has joined #openstack-infra01:46
*** Ryan_Lane has quit IRC01:46
jheskethFlorianSW: is it because no worker has registered for the job?01:49
FlorianSWjhesketh: sorry for the noob question: how i should add one? :/01:50
jheskethFlorianSW: you'll need to configure the job in jenkins and have it register the job with the gearman server01:51
jheskethI'm not sure how to do that exactly as I don't run a jenkins deployment sorry01:51
jheskethFlorianSW: If you telnet into the gearman server you show the registered functions and workers etc01:53
FlorianSWjhesketh: hmm, thanks so far, let me check it (i thought the gearman plugin itself is the worker for a job)01:54
jheskethFlorianSW: the gearman plugin is the worker, but the worker needs to register which functions it can perform01:54
jheskethpresumably that's just all the jobs by name in jenkins01:55
FlorianSWjhesketh: in gearman (command "workers") i see the host twice with _manager suffix01:56
jheskethFlorianSW: yeah, that's correct... the main worker registers for all the jobs, while the manager worker registers for other notifications from zuul (such as cancel the job)01:57
jheskethwhich the manager worker will then do internally etc01:57
FlorianSWjhesketh: and both named with manager?01:58
jheskethFlorianSW: as in you see two manager's?01:59
FlorianSWjhesketh: yes, two workers, both are named "servername_manager"01:59
FlorianSWand one zuul rpc listener01:59
jheskethhmm, that's odder... one might be a dead connection02:00
jhesketh(of which there is no harm as the jobs are cancelled by uuid)02:00
FlorianSWso not really a "problem" which prevents jenkins from doing his jobs, right?02:01
jheskethshouldn't be02:01
jheskethalthough you should see your jenkins worker (ie, a non-manager)02:01
jheskethbut again, I'm not familiar with the jenkins-gearman stuff so I may be wrong02:01
FlorianSWjhesketh: thanks for your help. maybe someone other can help, too :)02:03
jheskethno trouble, sorry I couldn't be of more help02:04
FlorianSWnp, this was better then nothing! :)02:04
*** Sukhdev has joined #openstack-infra02:05
*** nosnos has joined #openstack-infra02:10
*** signed8bit is now known as signed8bit_ZZZzz02:11
*** baoli has quit IRC02:12
*** ryanpetrello has quit IRC02:14
*** carl_baldwin has quit IRC02:23
*** oomichi has joined #openstack-infra02:23
FlorianSWjhesketh: just for info: After a restart of zuul and jenkins i have _exec as a new worker assigned to all jobs i have in my jenkins setup, but still the same error (jenkins doesn't add new builds)02:23
FlorianSWjhesketh: and another question: do you know a mailing list/forum/wikipage where i can ask for help?02:24
*** yaguang has joined #openstack-infra02:30
jheskethFlorianSW: do you mean zuul doesn't add new builds, or jenkins?02:32
jheskethFlorianSW: this channel, the openstack-dev mailing list, or openstack-infra02:32
*** baoli has joined #openstack-infra02:32
jheskethhttp://lists.openstack.org/02:32
FlorianSWjenkins, i think, at least if i understand the logs correctly02:33
*** hdd has joined #openstack-infra02:36
*** hichtakk has joined #openstack-infra02:42
*** lttrl has joined #openstack-infra02:48
*** otter768 has joined #openstack-infra02:51
*** hichtakk has quit IRC02:53
*** otter768 has quit IRC02:55
*** hdd has quit IRC02:59
*** achanda has joined #openstack-infra03:06
*** baoli has quit IRC03:09
*** dims has quit IRC03:10
*** jerryz has quit IRC03:12
*** dims has joined #openstack-infra03:13
*** achanda has quit IRC03:19
*** mtanino has quit IRC03:19
*** Masahiro has quit IRC03:26
*** signed8bit_ZZZzz has quit IRC03:29
openstackgerritKen'ichi Ohmichi proposed openstack/requirements: Cap a pbr version for a workaround  https://review.openstack.org/14335103:31
*** praveenkumar has joined #openstack-infra03:40
*** dkranz has quit IRC03:40
*** dims has quit IRC03:41
*** nosnos has quit IRC03:41
*** nosnos has joined #openstack-infra03:42
*** nosnos has quit IRC03:46
*** hichtakk has joined #openstack-infra03:54
*** hichtakk has quit IRC03:59
*** carl_baldwin has joined #openstack-infra04:01
*** stevemar has quit IRC04:01
*** stevemar has joined #openstack-infra04:02
openstackgerritKen'ichi Ohmichi proposed openstack/requirements: Cap a pbr version for a workaround  https://review.openstack.org/14335104:03
openstackgerritKen'ichi Ohmichi proposed openstack/requirements: Encap pbr versions for a workaround  https://review.openstack.org/14335104:04
*** Sukhdev has quit IRC04:05
*** hichtakk has joined #openstack-infra04:07
*** Sukhdev has joined #openstack-infra04:11
*** nosnos has joined #openstack-infra04:15
*** serverascode____ has quit IRC04:16
*** serverascode____ has joined #openstack-infra04:18
*** FlorianSW has quit IRC04:18
*** stevemar has quit IRC04:21
*** stevemar has joined #openstack-infra04:21
*** asettle-BRB is now known as asettle04:24
*** Masahiro has joined #openstack-infra04:27
*** hichtakk has quit IRC04:30
*** dimtruck is now known as zz_dimtruck04:30
*** hichtakk has joined #openstack-infra04:30
*** koolhead17 has joined #openstack-infra04:32
*** lttrl has quit IRC04:32
*** xyang1 has quit IRC04:33
*** hichtakk has quit IRC04:35
*** dims has joined #openstack-infra04:41
*** Ryan_Lane has joined #openstack-infra04:45
*** dims has quit IRC04:47
*** otter768 has joined #openstack-infra04:52
*** otter768 has quit IRC04:56
*** yamamoto_ has joined #openstack-infra04:59
*** koolhead_ has joined #openstack-infra05:03
*** koolhead17 has quit IRC05:04
*** _shaps__ has joined #openstack-infra05:04
*** armax has quit IRC05:04
*** sressot has quit IRC05:05
*** crinkle has quit IRC05:05
*** crinkle has joined #openstack-infra05:05
*** phschwartz has quit IRC05:05
*** armax has joined #openstack-infra05:06
*** xianghui has quit IRC05:06
*** xianghui has joined #openstack-infra05:06
*** _shaps_ has quit IRC05:06
*** phschwartz has joined #openstack-infra05:07
*** sressot has joined #openstack-infra05:07
*** Mmike has quit IRC05:09
*** spiffxp has quit IRC05:09
*** vhoward has quit IRC05:09
*** vhoward has joined #openstack-infra05:09
*** redrobot has quit IRC05:09
*** cinerama` has joined #openstack-infra05:10
*** yolanda_ has joined #openstack-infra05:10
*** notmyname_ has joined #openstack-infra05:10
*** therve` has joined #openstack-infra05:10
*** Guest20522 has quit IRC05:10
*** mtreinish has quit IRC05:11
*** notmyname has quit IRC05:11
*** dmsimard_away has quit IRC05:11
*** yolanda has quit IRC05:11
*** cinerama has quit IRC05:11
*** devananda has quit IRC05:11
*** therve has quit IRC05:11
*** devanand1 has joined #openstack-infra05:11
*** notmyname_ is now known as notmyname05:11
*** Guest20522 has joined #openstack-infra05:11
*** yamamoto_ has quit IRC05:11
*** mtreinish has joined #openstack-infra05:12
*** redrobot has joined #openstack-infra05:12
*** redrobot is now known as Guest8746305:12
*** spiffxp has joined #openstack-infra05:12
*** Mmike has joined #openstack-infra05:12
*** dmsimard_away has joined #openstack-infra05:13
*** dmsimard_away is now known as dmsimard05:13
openstackgerritJeremy Stanley proposed openstack-dev/pbr: Merge tag '0.10.7' into master  https://review.openstack.org/14336305:27
fungii think i'm done hacking on pbr for the night. known regressions are addressed in 0.10.7 as best i can tell05:38
*** hichtakk has joined #openstack-infra05:39
fungialso the setuptools 8 stack targeting pbr master has all regressions from the feature/0.10 branch addressed if they were present05:39
*** koolhead_ has quit IRC05:41
oomichifungi: thanks for pbr 0.10.7. by using it, tempest gate tests seems fixed :-)05:44
*** hdd has joined #openstack-infra05:45
*** krtaylor has joined #openstack-infra05:56
*** Mmike has quit IRC05:57
*** Mmike has joined #openstack-infra05:57
*** oomichi has quit IRC05:59
*** carl_baldwin has quit IRC06:02
*** yamamoto_ has joined #openstack-infra06:04
*** stevemar has quit IRC06:06
*** Sukhdev has quit IRC06:11
*** teran has joined #openstack-infra06:25
*** teran has quit IRC06:25
*** teran has joined #openstack-infra06:28
openstackgerritPraveen Kumar proposed openstack-infra/jenkins-job-builder: Add Example to demonstrate flow plugin for job-template  https://review.openstack.org/14325006:35
*** teran_ has joined #openstack-infra06:38
*** teran has quit IRC06:40
*** yfried|afk has joined #openstack-infra06:41
*** garyk has joined #openstack-infra06:44
*** teran_ has quit IRC06:45
*** carl_baldwin has joined #openstack-infra06:45
*** camunoz is now known as camunoz_gone06:45
*** [HeOS] has quit IRC06:46
*** carl_baldwin has quit IRC06:52
*** otter768 has joined #openstack-infra06:52
*** teran has joined #openstack-infra06:56
*** otter768 has quit IRC06:57
*** hdd has quit IRC06:59
*** yfried|afk is now known as yfried07:08
*** _nadya_ has joined #openstack-infra07:14
*** zz_dimtruck is now known as dimtruck07:18
*** teran_ has joined #openstack-infra07:19
*** e0ne has joined #openstack-infra07:21
*** teran has quit IRC07:22
*** _nadya_ has quit IRC07:22
*** teran has joined #openstack-infra07:27
*** yfried is now known as yfried|afk07:27
*** teran_ has quit IRC07:29
*** k4n0 has joined #openstack-infra07:30
*** _nadya_ has joined #openstack-infra07:33
*** salv-orlando has joined #openstack-infra07:33
*** ihrachyshka has joined #openstack-infra07:34
*** fabo_ is now known as fabo07:36
*** ihrachyshka has quit IRC07:38
*** ihrachyshka has joined #openstack-infra07:39
*** ihrachyshka has quit IRC07:41
*** hichtakk has quit IRC07:42
*** yfried|afk is now known as yfried07:47
*** subscope has joined #openstack-infra07:48
*** zz_avozza is now known as avozza07:48
*** yamamoto_ has quit IRC08:01
*** jgallard_ has joined #openstack-infra08:01
*** dimtruck is now known as zz_dimtruck08:06
*** zz_dimtruck is now known as dimtruck08:07
*** gilliard_afk is now known as gilliard08:15
*** jamielennox is now known as jamielennox|away08:15
*** teran has quit IRC08:17
*** dimtruck is now known as zz_dimtruck08:18
*** [HeOS] has joined #openstack-infra08:23
*** dorry has quit IRC08:23
*** _nadya_ has quit IRC08:25
*** HeOS has joined #openstack-infra08:26
*** [HeOS] has quit IRC08:26
*** ildikov has quit IRC08:26
*** e0ne has quit IRC08:27
*** ihrachyshka has joined #openstack-infra08:28
*** dmellado has quit IRC08:29
*** yamamoto_ has joined #openstack-infra08:31
*** garyk has quit IRC08:35
*** jerryz has joined #openstack-infra08:36
*** zz_dimtruck is now known as dimtruck08:37
*** e0ne has joined #openstack-infra08:40
praveenkumarIs gerrit/zuul service down, I am getting http://logs.openstack.org/50/143250/2/check/gate-jenkins-job-builder-pep8/398bbfc/console.html for a patch set08:44
*** e0ne has quit IRC08:44
*** dimtruck is now known as zz_dimtruck08:48
*** ZZelle has joined #openstack-infra08:50
*** dims has joined #openstack-infra08:50
*** otter768 has joined #openstack-infra08:53
*** garyk has joined #openstack-infra08:54
*** dims has quit IRC08:54
*** amuller has joined #openstack-infra08:54
*** Hal_ has joined #openstack-infra08:54
*** otter768 has quit IRC08:58
*** jpich has joined #openstack-infra08:58
*** kedkul has joined #openstack-infra09:01
*** Ala has joined #openstack-infra09:04
*** salv-orlando has quit IRC09:05
*** teran has joined #openstack-infra09:05
*** yfried is now known as yfried|afk09:06
*** yfried|afk is now known as yfried09:09
*** derekh has joined #openstack-infra09:13
*** kashyap has joined #openstack-infra09:13
*** fandi has joined #openstack-infra09:15
*** isaacb has joined #openstack-infra09:16
*** subscope has quit IRC09:16
*** garyk has quit IRC09:17
isaacbDoes anyone knows how to configure Zuul to combine several gerrit accounts with same jenkins master?09:18
*** eikke has joined #openstack-infra09:22
*** garyk has joined #openstack-infra09:23
eikkeany known issues with Gerrit (both web and git)?09:23
*** andreykurilin_ has joined #openstack-infra09:24
*** jedimike has joined #openstack-infra09:28
ZZelleeikke, are you having a dns resolution trouble?09:28
*** andreykurilin_ has quit IRC09:30
*** andreykurilin__ has joined #openstack-infra09:30
*** koolhead17 has joined #openstack-infra09:30
*** e0ne has joined #openstack-infra09:34
*** pblaho_ has joined #openstack-infra09:35
*** koolhead17 has quit IRC09:44
*** unicell has quit IRC09:44
*** yaguang has quit IRC09:45
*** Longgeek has joined #openstack-infra09:51
*** salv-orlando has joined #openstack-infra09:54
*** AJaeger has joined #openstack-infra09:59
*** garyk has quit IRC10:03
*** garyk has joined #openstack-infra10:04
*** e0ne is now known as e0ne_10:05
eikkeZZelle: at first DNS, afterwards connection ("network is unreachable" to port 29418, but other sites etc worked fine)), now DNS again10:06
ZZelleeikke, i got DNS trouble ... solved by using 8.8.8.8 google DNS10:07
ZZelleeikke, but no connection troubles10:07
*** jedimike has quit IRC10:09
*** koolhead17 has joined #openstack-infra10:10
*** koolhead17 has joined #openstack-infra10:10
openstackgerritDmitry Teselkin proposed openstack-infra/zuul: Process event multiple times.  https://review.openstack.org/13602710:13
*** e0ne_ has quit IRC10:16
*** [HeOS] has joined #openstack-infra10:16
*** HeOS has quit IRC10:17
*** kashyap has quit IRC10:17
*** hughsaunders has quit IRC10:18
*** kashyap has joined #openstack-infra10:18
*** hughsaunders has joined #openstack-infra10:18
*** Masahiro has quit IRC10:21
*** Masahiro has joined #openstack-infra10:21
*** yolanda_ has quit IRC10:22
*** BobBall_1WOL is now known as BobBall10:22
*** _nadya_ has joined #openstack-infra10:24
*** andreykurilin__ has quit IRC10:25
openstackgerritRadomir Dopieralski proposed openstack-infra/project-config: Update stackforge/tuskar-ui-extras to run tests on the gate  https://review.openstack.org/14304310:27
*** garyk has quit IRC10:29
*** AJaeger has quit IRC10:34
*** garyk has joined #openstack-infra10:35
*** Masahiro has quit IRC10:37
*** cnesa has joined #openstack-infra10:37
*** sabarij is now known as zz_sabarij10:43
*** rfolco has joined #openstack-infra10:45
*** yolanda has joined #openstack-infra10:46
*** rlandy has joined #openstack-infra10:50
*** pblaho_ has quit IRC10:51
*** isaacb has quit IRC10:54
*** otter768 has joined #openstack-infra10:54
*** otter768 has quit IRC10:59
*** tristanC has quit IRC11:01
*** _nadya_ has quit IRC11:02
*** tristanC has joined #openstack-infra11:02
*** yfried is now known as yfried|afk11:03
*** tristanC has quit IRC11:06
*** jgallard_ has quit IRC11:06
*** yfried|afk is now known as yfried11:07
*** tristanC has joined #openstack-infra11:07
*** unicell has joined #openstack-infra11:08
*** garyk has quit IRC11:08
*** nosnos has quit IRC11:13
*** nosnos has joined #openstack-infra11:13
*** redixin has joined #openstack-infra11:14
*** nosnos has quit IRC11:17
*** Masahiro has joined #openstack-infra11:18
*** Masahiro has quit IRC11:20
*** koolhead17 has quit IRC11:27
*** e0ne has joined #openstack-infra11:28
*** yfried is now known as yfried|afk11:32
*** jedimike has joined #openstack-infra11:33
*** lttrl has joined #openstack-infra11:37
*** zz_dimtruck is now known as dimtruck11:38
*** isaacb has joined #openstack-infra11:42
*** dizquierdo has joined #openstack-infra11:42
*** lttrl has quit IRC11:42
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/storyboard: Implement Streaming API websocket handler  https://review.openstack.org/14330411:42
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/storyboard: Implement Streaming API websocket handler  https://review.openstack.org/14330411:44
*** dims has joined #openstack-infra11:45
*** achanda has joined #openstack-infra11:46
*** garyk has joined #openstack-infra11:47
*** koolhead17 has joined #openstack-infra11:48
*** dimtruck is now known as zz_dimtruck11:48
*** tnurlygayanov has joined #openstack-infra11:52
*** yfried|afk is now known as yfried11:52
*** adalbas has joined #openstack-infra11:54
*** e0ne is now known as e0ne_11:57
*** adalbas has quit IRC11:58
*** dims has quit IRC11:58
*** wuhg has joined #openstack-infra12:00
*** avozza is now known as zz_avozza12:02
*** zz_avozza is now known as avozza12:02
*** fifieldt__ has quit IRC12:02
*** _nadya_ has joined #openstack-infra12:02
*** adalbas has joined #openstack-infra12:06
wuhghi, all , i have a question on gerrit ,anybody here?12:06
*** fifieldt has joined #openstack-infra12:07
*** e0ne_ has quit IRC12:07
*** mase_x200 has joined #openstack-infra12:08
*** koolhead17 has quit IRC12:08
*** _nadya_ has quit IRC12:09
*** ddieterl_ has joined #openstack-infra12:14
*** ddieterly has quit IRC12:15
dstufftfungi: clarkb mordred just FYI, I'm cutting pip 6 later today12:15
*** cdent has joined #openstack-infra12:16
*** SumitNaiksatam has quit IRC12:16
*** e0ne has joined #openstack-infra12:16
*** dims has joined #openstack-infra12:18
*** SumitNaiksatam has joined #openstack-infra12:20
*** jgjl has joined #openstack-infra12:22
*** denis_makogon has joined #openstack-infra12:22
openstackgerritAleksey proposed openstack-infra/storyboard: Changes for calls iteritems and str to six  https://review.openstack.org/14273312:23
*** koolhead17 has joined #openstack-infra12:23
*** achanda has quit IRC12:32
*** achanda has joined #openstack-infra12:33
*** baoli has joined #openstack-infra12:33
*** baoli has quit IRC12:34
*** baoli has joined #openstack-infra12:34
*** achanda has quit IRC12:37
*** dims_ has joined #openstack-infra12:39
*** kedkul has quit IRC12:42
*** dims has quit IRC12:42
*** yfried is now known as yfried|afk12:44
*** FlorianSW has joined #openstack-infra12:49
*** pblaho_ has joined #openstack-infra12:53
*** pc_m has joined #openstack-infra12:53
*** jgallard_ has joined #openstack-infra12:54
*** otter768 has joined #openstack-infra12:55
*** mbacchi has joined #openstack-infra12:57
*** baoli has quit IRC12:59
*** e0ne is now known as e0ne_13:00
*** otter768 has quit IRC13:00
*** yfried|afk is now known as yfried13:00
*** pblaho_ has quit IRC13:01
*** jgjl has quit IRC13:01
*** e0ne_ is now known as e0ne13:02
*** tobiash has joined #openstack-infra13:03
*** subscope has joined #openstack-infra13:05
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/storyboard: Implement Streaming API websocket handler  https://review.openstack.org/14330413:06
*** dpyzhov has joined #openstack-infra13:06
*** Longgeek has quit IRC13:10
*** subscope has quit IRC13:10
*** dkranz has joined #openstack-infra13:11
*** dprince has joined #openstack-infra13:13
*** dims_ has quit IRC13:13
*** Longgeek has joined #openstack-infra13:14
*** yfried has quit IRC13:14
*** jgjl has joined #openstack-infra13:15
*** yamamoto_ has quit IRC13:16
*** yamamoto has joined #openstack-infra13:17
*** dims has joined #openstack-infra13:17
*** bswartz has quit IRC13:17
*** avozza is now known as zz_avozza13:18
*** e0ne has quit IRC13:26
*** hashar has joined #openstack-infra13:26
*** e0ne has joined #openstack-infra13:26
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/storyboard: Implement Streaming API websocket handler  https://review.openstack.org/14330413:27
*** pblaho_ has joined #openstack-infra13:27
rcarrillocruzhmm13:27
rcarrillocruzi'm getting random dns resolve issues on review.openstack.org13:27
rcarrillocruzanyone else?13:27
FlorianSWrcarrillocruz: me too, for openstack.org itself, too13:28
*** Masahiro has joined #openstack-infra13:29
*** jokke__ is now known as jokke_13:30
*** Masahiro_ has joined #openstack-infra13:32
*** Masahiro has quit IRC13:34
*** lttrl has joined #openstack-infra13:34
*** hashar has quit IRC13:35
*** Masahiro has joined #openstack-infra13:35
openstackgerritNikita Konovalov proposed openstack-infra/python-storyboardclient: Setting up base classes  https://review.openstack.org/13809213:36
*** Masahiro_ has quit IRC13:36
*** Longgeek has quit IRC13:38
*** kgiusti has joined #openstack-infra13:38
*** Longgeek has joined #openstack-infra13:38
*** 16WAAORL2 has quit IRC13:40
*** enikanorov_ has joined #openstack-infra13:40
*** Masahiro has quit IRC13:40
*** Masahiro has joined #openstack-infra13:41
*** ayoung has joined #openstack-infra13:42
*** _nadya_ has joined #openstack-infra13:49
*** Masahiro has quit IRC13:49
*** e0ne is now known as e0ne_13:52
*** Masahiro has joined #openstack-infra13:54
*** bswartz has joined #openstack-infra13:56
*** Masahiro has quit IRC13:58
anteayaI can navigate to review.o.o and sign in successfully13:58
anteayawhat actions are creating issues13:58
*** Masahiro has joined #openstack-infra13:58
anteayazuul appears to be functioning according to my scan of status.o.o/zuul14:00
openstackgerritMerged openstack-infra/storyboard-webclient: Added millisecond delay to reduce the number of queries.  https://review.openstack.org/14012014:01
openstackgerritCedric Brandily proposed openstack-infra/git-review: Handle correctly http error raise in run_http_exc  https://review.openstack.org/12439614:01
anteayapraveenkumar: that log is showing a failure to connect to git14:01
* anteaya checks git.o.o14:01
*** Masahiro has quit IRC14:03
*** Masahiro has joined #openstack-infra14:03
*** lttrl has quit IRC14:05
*** baoli has joined #openstack-infra14:05
*** pblaho_ has quit IRC14:06
anteayaI just successfully cloned jenkins-job-builder from all five git servers14:07
anteayaperhaps there was an issue with the front end14:07
anteayawuhg: what is your question?14:08
anteayaSergeyLukjanov fungi clarkb the meet bot seems to be able to start a meeting but not to change topics14:09
anteayaI was looking in -meeting14:10
*** yamamoto has quit IRC14:11
dhellmannfungi: thanks for pushing through the rest of that pbr work -- I need to catch up on commit logs14:11
wuhganteaya: why didn't the new patch https://review.openstack.org/#/c/110425/ refresh/empty the review ?14:12
*** yfried has joined #openstack-infra14:12
*** mase_x200 has quit IRC14:12
*** Masahiro has quit IRC14:12
wuhganteaya: the Code-Review column should be empty after a new patch ,right?14:13
anteayawuhg: what was the change in the new patch?14:13
openstackgerritMerged openstack-infra/storyboard-webclient: Users may now issue their own access tokens.  https://review.openstack.org/14017614:14
*** e0ne_ is now known as e0ne14:14
wuhgpatch set 4914:14
*** Guest32161 has joined #openstack-infra14:15
anteayawhat was changed?14:15
dhellmannfungi, clarkb : here's an interesting note from gerrit: https://review.openstack.org/#/c/142593/1 says it is both merged and in merge conflict. Is that because it's on a branch?14:15
*** Masahiro has joined #openstack-infra14:15
anteayaif just the commit message was changed then no, the current review votes will stay14:16
*** fandi has quit IRC14:16
*** fandi_ has joined #openstack-infra14:16
anteayaif code was changed then yes, the code review votes for the prior patchset should be cleared14:16
anteayawuhg: was there a code change between patchset 48 and 49?14:16
*** mfer has joined #openstack-infra14:18
anteayathere is so much code there, I can't tell14:18
*** leakypipes is now known as jaypipes14:18
*** Masahiro has quit IRC14:20
*** signed8bit has joined #openstack-infra14:23
*** signed8bit has quit IRC14:23
crinkleI'm also seeing dns issues: I can't resolve any *.o.o from my home, and there are jobs failing because of dns issues http://logs.openstack.org/85/143385/1/check/gate-puppet-keystone-puppet-unit-3.6/dac1786/console.html#_2014-12-22_07_36_58_10614:24
*** xyang1 has joined #openstack-infra14:24
crinklealso, interestingly, I can't resolve rackspace.com14:24
*** dims has quit IRC14:26
wuhganteaya:it seems it's rebased14:26
anteayawuhg: well for a rebase I believe the votes stay14:27
openstackgerritAleksey proposed openstack-infra/storyboard: Git repo links in project table  https://review.openstack.org/14345514:28
anteayasince the system I believe can recognize a rebase as not a change in the code in the patch and therefore the reviews stay14:28
anteayacrinkle: thanks for the additional information14:28
anteayaJayF: are you about?14:28
crinkleanteaya: np14:28
anteayaI can't reproduce so having your perspective is valuable14:29
wuhganteaya thanks14:29
anteayawuhg: thank you14:29
anteayathere was a spike in errors accross all hosts but currently the situation seems to have resolved14:30
*** dims has joined #openstack-infra14:31
anteayacrinkle: you are currently seeing dns issues?14:31
anteayathat log was from 7 hours agao14:31
anteayaperhaps there was a blip at that time14:31
crinkleanteaya: yes, still unable to resolve review.openstack.org from my home (but able to from elsewhere)14:32
anteayacrinkle: :(14:33
crinkleit could perhaps just be my isp14:33
anteayawhich isp are you with?14:33
crinklecentury link14:33
jesusauruscrinkle: strange, is there anything in the authority section when you run 'dig openstack.org'?14:34
*** mikeit has joined #openstack-infra14:34
crinklejesusaurus: nope14:34
jesusaurus:(14:34
anteayacrinkle: I've kind of exhuasted my repertoire until fungi or other show up14:36
anteayamy normal mode of operation is to ask questions until someone who has a clue and can do something arrives14:36
crinkleanteaya: np, I've hacked my /etc/hosts for now14:36
anteayacrinkle: okay14:37
anteayacrinkle: thanks for being a data point14:37
crinklehappy to help14:37
anteaya:D14:37
*** zz_avozza is now known as avozza14:39
*** eharney has joined #openstack-infra14:43
anteayacrinkle: so the meetbot was able to start and end the meeting but wouldn't accept a topic I see14:44
* anteaya looks for logs14:44
anteayayou got logs: http://eavesdrop.openstack.org/meetings/puppet_openstack/2014/puppet_openstack.2014-12-22-14.05.log.html14:45
crinkleanteaya: yes, and it seemed like the #link command worked14:45
crinklewe didn't really exercise the other commands14:45
crinklejust #topic was behaving strange14:46
anteayaI see yes14:46
*** mikeit has quit IRC14:46
anteayaodd that it could hear the startmeeting and endmeeting commands but not topic14:46
anteayanote that I am not issuing those commands in this channel14:46
anteaya:D14:46
crinklehaha14:46
anteayameetbot is also used for logging channels14:47
nibalizermaybe just because the bot doesn't have ops right now14:47
anteayabut well helpfully start a meeting anywhere14:47
anteayanibalizer: is that it14:47
anteayafunny it can still start and end meetings though14:48
nibalizerit probably issued the topic commands on que14:48
anteayaokay so an easy fix for someone to bless it with ops14:48
nibalizerjust freenode didn't care because it doesn't have ops14:48
nibalizerer cue?14:48
nibalizerhow english14:48
*** avozza is now known as zz_avozza14:49
openstackgerritAlexandre Viau proposed openstack-infra/project-config: python-surveilclient: Added py34 checks  https://review.openstack.org/14346014:54
*** otter768 has joined #openstack-infra14:56
*** mfer has quit IRC14:56
*** adalbas has quit IRC14:57
fungipraveenkumar: eikke: ZZelle: rcarrillocruz: crinkle: anteaya: https://status.rackspace.com/ is always a good place to look for these sorts of things14:57
fungi"We are monitoring a connectivity issue impacting our DNS environment that began at approximately 01:10 CST. Our engineers have taken steps to mitigate impact from this issue, however some of our customers may continue to experience intermittent periods of latency, packet loss, or connectivity failures when attempting to reach rackspace.com or subdomains within rackspace.com..."14:57
anteayafungi: thanks14:58
fungi(of course, if you can't resolve status.rackspace.com either, then not terribly helpful admittedly)14:58
*** mjturek has joined #openstack-infra14:58
anteayabut I could have posted to a paste in that case14:58
anteayahad I looked which I didn't14:58
*** beekneemech is now known as bnemec14:58
*** jgjl has quit IRC14:59
fungianteaya: nibalizer: yeah when freenode was doing stuff over the weekend the meetbot presumably reconnected while chanserv and other atheme services were split/offline so never got auto-op'ed. i'll fix14:59
anteayathanks14:59
*** yolanda has quit IRC15:00
*** amitgandhinz has joined #openstack-infra15:00
*** yolanda has joined #openstack-infra15:00
*** otter768 has quit IRC15:01
yolandahere, fungi or infra-cores, we are having some issues with storyboard server15:01
yolandalooks as new changes aren't propagated, we don't see updates on code since friday at least15:02
fungiyolanda: i'll check into it15:02
yolandathx15:02
fungipuppetboard says "80 nodes unreported in the last 0.5 hours" and seems to stretch back 3-5 days... that's no good15:03
yolandaany nodes, not only storyboard?15:04
fungiyolanda: looking that way. 80 out of 90 nodes unresponsive15:05
ZZellefungi, thanks for reading the backlog :)15:05
yolandafungi, they are on holiday :)15:05
yolandaok, need to leave for a while, i'll be back on an hour15:05
*** bhunter71 has joined #openstack-infra15:06
bhunter71#join git-upstream15:06
bhunter71^ excuse me15:07
anteayabhunter71: it happens15:08
anteayadhellmann: so I am reviewing https://review.openstack.org/#/c/142257/ would appreciate having your input15:08
*** sigmavirus24_awa is now known as sigmavirus2415:08
anteayaI haven't been following all the details (I've tried, I can't keep up) but you seemed to have a voice in the conversation about pbr freeze15:09
mordredfungi: didn't we recently land that change to ansible running?15:13
fungimordred: which change?15:13
fungimordred: the one to switch from the static hostlist to puppet cert list --all15:13
fungi?15:13
mordredNo15:13
*** dpyzhov has left #openstack-infra15:13
fungimordred: oh, if it was something in the past few days, then likely15:13
mordredThe one about running even if git fails15:14
*** mfer has joined #openstack-infra15:15
*** ryanpetrello has joined #openstack-infra15:16
*** Masahiro has joined #openstack-infra15:16
*** mfer has quit IRC15:19
dhellmannanteaya: looking15:19
*** jerryz has quit IRC15:19
anteayadhellmann: thank you, just didn't want to catch you unaware15:19
*** tonytan4ever has joined #openstack-infra15:20
dhellmannanteaya: thanks. +1 from me, with a comment about future cleanup15:20
anteayadhellmann: thank you15:20
*** Masahiro has quit IRC15:21
fungimordred: looks like you're talking about https://review.openstack.org/129722 but that merged ~10 days ago15:21
mordredAh. K15:22
*** stevemar has joined #openstack-infra15:23
*** annegentle has joined #openstack-infra15:24
openstackgerritAleksey proposed openstack-infra/storyboard: All aborts now have message  https://review.openstack.org/14346515:25
*** ayoung has quit IRC15:28
*** isaacb has quit IRC15:34
*** ryanpetrello has quit IRC15:34
*** mtanino has joined #openstack-infra15:35
*** teran has quit IRC15:38
*** teran has joined #openstack-infra15:38
fungimordred: clarkb: https://review.openstack.org/129441 merged at exactly the right time to account for this "Bail out in run_all.sh runs when failures happen"15:40
fungii wonder if we're bailing out before updating most of the servers15:40
*** zz_avozza is now known as avozza15:40
fungichecking the ansible log, it doesn't mention any of the servers which are showing up as unreported, and this would also explain why only 80 of the 90 are not reporting rather than all of them15:41
fungiaha! i see why15:42
fungiso, if we take this route, we can't have _any_ servers with puppet certs not getting updated via ansible15:43
mordredfungi: yah. sorry I didn't see that patch before15:43
mordredI recommend making it +e again before we start running ansible15:43
fungiafstest.openstack.org has a cert, but we intentionally didn't accept its ssh key on the puppetmaster because we didn't want it getting updated15:43
openstackgerritMerged openstack-infra/project-config: Move to pbr freeze from pip freeze  https://review.openstack.org/14225715:43
fungiso when it gets to that point in the list, it fails to connect to afstest and then aborts the remainder of the run15:44
mordredwait, really?15:44
fungii'll whip up the one-liner patch to set +e right before the ansible loop15:44
mordredfungi: hang on - that doesn't make sense to me15:44
fungimordred: the last line in the log for each invocation of run_all.sh is "afstest.openstack.org      : ok=0    changed=0    unreachable=1    failed=0"15:45
fungierror detail is "fatal: [afstest.openstack.org] => SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue"15:45
mordredfungi: ah - it's because the afs things are happening before "everything else15:46
mordredsorry15:46
mordredI had a stale copy of the file locally15:46
mordredyes - +e before the ansible section and we should be back to sane15:46
fungiawesome--thanks!15:46
*** EmilienM is now known as EmilienM|afk15:47
mordredooh! the pbr freeze change landed15:47
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Don't bail out in run_all.sh for ansible errors  https://review.openstack.org/14347415:50
fungimordred: clarkb: ^15:50
fungithat should get puppet working again i think15:50
*** achanda has joined #openstack-infra15:51
dstufftfungi: mordred pip 6 in either 15 minutes, or an hour15:52
dstufftdepending on if I feel like doing it in 1515:52
fungidstufft: excellent. i guess i'll clear my schedule for the day ;)15:52
dstufftfungi: you cut me sir15:52
dstufft:D15:52
dstuffthopefully pip 6.0 will be relatively pain free, pep 440 is the main thing15:53
* dstufft waited until everyone solved pep 440 with setuptools so nobody gets mad at pip15:53
mordreddstufft: smart15:53
mordreddstufft: well, in 15 minutes I'm walking over to Roscoe's Fried Chicken and Waffles - so I will not care15:54
dstufftmordred: also smart15:54
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration to add test attrs column to test_runs table  https://review.openstack.org/14323915:54
dstufftI wish I had some fried Chicken and waffles now15:54
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Switch subunit2sql to set attrs in test_runs table  https://review.openstack.org/14324015:54
fungiwell, i think i finally got the last of the fallout from the setuptools pep 440 implementation solved in released versions of pbr last night, so good timing15:55
omrimIs someone know what;s wrong with "install_puppet.sh" script?15:56
anteayaomrim: we are having a third-party meeting right now15:56
fungiomrim: i think we'd need more context than that. are you experiencing a problem with it?15:56
anteayaomrim: you might consider stopping by15:56
*** ihrachyshka has quit IRC15:56
*** rlandy has quit IRC15:57
*** hdd has joined #openstack-infra15:57
omrimSorry to interrupt, Yes I have an issue with the script15:58
*** baoli has quit IRC15:58
*** julim has joined #openstack-infra15:59
krotscheckStoryboard meeting in #openstack-meeting-316:00
anteayaomrim: the third party meeting is over, it is a perfect place for questions like yours, there are three to choose from so you can get questions answered and logged so you can help others in future16:01
*** ryanpetrello has joined #openstack-infra16:02
anteayaomrim: if you want any help you are going to have to share your output in a paste16:02
anteayaomrim: we don't know enough about what you are asking to help you16:02
fungiomrim: http://www.catb.org/esr/faqs/smart-questions.htmlhttp://www.catb.org/esr/faqs/smart-questions.html16:02
fungiomrim: er, sorry, my paste button bounced16:02
anteayaomrim: and I agree with fungi, take some time and read that page16:02
fungiomrim: we need to know what error you're seeing to have any hope of helping you16:02
*** dkranz has quit IRC16:03
*** ayoung has joined #openstack-infra16:04
omrimIt is quite simple..I just try to install a new Jenkins master. So I take the install_master.sh from: https://raw.github.com/jaypipes/os-ext-testing/master/puppet/install_master.sh16:05
*** bradjones has joined #openstack-infra16:05
openstackgerritJames Polley proposed openstack-infra/project-config: Don't run the python jobs, except for pep8 testing  https://review.openstack.org/14347916:05
*** prad has joined #openstack-infra16:05
omrimWhen I run the install_master the script got an exception in puppet instllation section16:05
*** prad has quit IRC16:06
openstackgerritJames Polley proposed openstack-infra/project-config: Don't run the python jobs, except for pep8 testing  https://review.openstack.org/14347916:06
omrimWhen I look in the code I so the following: http://paste.openstack.org/show/153915/16:06
*** pblaho_ has joined #openstack-infra16:06
*** _nadya_ has quit IRC16:07
yolandafungi, checking the logs, have you found the issue? something with ansible?16:07
fungiomrim: that url is no longer valid16:07
*** dannywilson has joined #openstack-infra16:07
omrimSo we need to update the install_master script right?16:08
omrimI took it from the master16:08
fungiomrim: you should use https://git.openstack.org/cgit/openstack-infra/system-config/plain/install_puppet.sh16:09
fungiomrim: we don't maintain the install_master script you're talking about. that's just something someone wrote long ago and stuck on github16:09
fungiomrim: it's not anything even remotely official or supported by the openstack infrastructure team16:10
*** mfer has joined #openstack-infra16:11
*** ryanpetrello has quit IRC16:11
omrimfungi: That's correct..Thanks16:11
fungiomrim: as such, i don't have control to update it16:11
*** dannywilson has quit IRC16:12
*** e0ne is now known as e0ne_16:12
*** dannywilson has joined #openstack-infra16:15
*** koolhead17 has quit IRC16:16
fungii'm going to self approve https://review.openstack.org/143474 since mordred +2'd it in spirit ;)16:16
fungithat way we'll start puppeting our servers again16:17
mordred++16:17
fungiespecially important since we've got pip 6 on the way and would like to be able to see what's breaking (if anything)16:18
mordredYah16:18
yolandafungi, great, thx16:18
*** dkranz has joined #openstack-infra16:19
*** e0ne_ is now known as e0ne16:19
mordredBTW, I just had an idea on making notifications better ...16:19
*** mfer has quit IRC16:19
mordredI'll write it down when I'm on the laptop16:20
fungidhellmann: if you end up getting a few minutes, 142931 and the changes on top of it should get pbr master back on track with setuptools 8 now16:21
jaypipesfungi: is there a URL you would like me to point people to that explains how to install an upstream CI clone easily?16:21
openstackgerritMerged openstack-infra/system-config: Don't bail out in run_all.sh for ansible errors  https://review.openstack.org/14347416:22
fungijaypipes: we have documentation on how to install the various components. hoping someone gets time to contribute a maintainable document for that specific use case however16:22
*** avozza is now known as zz_avozza16:22
*** carl_baldwin has joined #openstack-infra16:23
*** thedodd has joined #openstack-infra16:25
jaypipesfungi: it's not really about documentation on how to install individual components. what people need is instructions on how to install the entire thing in an automated way, especially the slave nodes. And.. how to keep such a thing up-to-date when the upstream Puppet or other configuration stuff changes.16:26
jeblairjaypipes: https://review.openstack.org/#/c/139745/216:27
*** enikanorov_ has quit IRC16:27
jaypipesjeblair: nice.16:28
jesusaurusjaypipes: yeah, that last bit can be very challenging16:28
jaypipesyup.16:28
fungilooks like https://etherpad.openstack.org/p/third-party-ci-documentation has an outline coming together16:28
jeblairjaypipes: yeah, i think we're getting to a point in the puppet refactoring that we can start to seriously think about that, and there are third-party folks showing up to help with the work and keep it up to date16:28
*** _nadya_ has joined #openstack-infra16:28
* jesusaurus just lost dns to review.openstack.org :(16:30
*** dizquierdo has quit IRC16:30
fungijaypipes: and while the infra team does write a lot of documentation already, we didn't really mandate third-party testing (it was various openstack project review teams who independently decided they wanted it) so hoping to see documentation effort emerge from the same places the requirements did16:30
fungijesusaurus: see status.rackspace.com16:31
*** dims has quit IRC16:31
*** dims has joined #openstack-infra16:31
jesusaurusfungi: yeah, i saw that chatter earlier, it still makes me sad16:31
*** koolhead17 has joined #openstack-infra16:32
*** teran has quit IRC16:32
jaypipesfungi: I was just trying to be helpful when I created the os-ext-testing thing and the blog posts. I wasn't trying to make your life harder.16:33
fungijaypipes: i didn't think you were! but it's a moving target16:33
fungijaypipes: so for any documentation effort to be useful long-term it needs to be tied to the same places the software it's tracking is developed16:34
jaypipesfungi: I recognize that it has. I've put a disclaimer/warning up on the blog posts. And I will try to push some docs in the holidays to the spec linked above.16:34
jaypipesfungi: agreed.,16:34
dstufftfungi: mordred fwiw pip 6 is out now, virtualenv will be following soon16:34
dstufftand then the actual release announcement16:34
*** pcrews has joined #openstack-infra16:34
fungidstufft: thanks! i'm looking forward to all the new awesome16:34
fungijaypipes: and given what you wrote is published under an open license, it makes a good place for more official documentation to get cut and pasted from (with appropriate credit/citation), so i don't want you to feel like the effort was at all wasted16:35
*** carl_baldwin has quit IRC16:36
*** carl_baldwin has joined #openstack-infra16:39
omrimfungi: After a succes insalaation of the puppet i Got the error: Could not find template 'openstack_project/puppet.conf.erb'16:40
omrimfungi: What did I miss?16:40
dhellmannfungi: we had to collapse a bunch of changes, it seems?16:40
*** ryanpetrello has joined #openstack-infra16:41
fungidhellmann: yep, they were independently necessary to support setuptools 8 and to be properly self-testing16:41
fungidhellmann: the ones which weren't interdependent are children of that change16:42
dhellmannfungi: sounds good, I'll scan those now16:42
fungidhellmann: but that squash ended up being necessary to get the change to pass all its jobs on master16:42
dhellmannthat's what I figured16:42
openstackgerritMerged openstack-infra/project-config: Add new Cloud-Pydashie project  https://review.openstack.org/14224016:43
fungiit was ugly ans i wish there were a better way, but it is what it is16:43
*** sputnik13 has joined #openstack-infra16:43
fungiomrim: no idea. that template definitely exists in the openstack-infra/system-config repository16:43
*** zz_avozza is now known as avozza16:44
fungiomrim: i'd need to see that error in much more context to have any idea why you might be getting it16:44
*** reed has joined #openstack-infra16:45
jeblairfungi: http://phenomena.nationalgeographic.com/2014/12/19/when-threatened-by-worms-bacteria-summon-killer-fungi/16:46
omrimhttp://paste.openstack.org/show/153924/16:46
dhellmannfungi: ugh, looks like the pbr patches are all failing16:46
dstufftfungi: as always, let me know if things go sideways16:47
fungithe ansible run which just started is now using the updated script with set +e and has progressed past the broken server, so we should see stuf coming current on puppetboars in the next few minutes16:47
fungidstufft: i will be sure to16:47
clarkbkiller fungi!16:47
fungidhellmann: eh?16:47
anteayaI am currently operating under the belief that the fungi username isn't symbolizing mushrooms but the latin "to produce"16:47
pleia2morning16:47
* anteaya waits to be corrected16:47
anteayamorning pleia216:47
fungidhellmann: might be we missed our window of opportunity to merge those before pip 6 released a few minutes ago16:48
dstufftI just assumed fungi really loved shrooms16:48
anteayadstufft: I would not argue with that possibility16:48
*** baoli has joined #openstack-infra16:48
clarkbfungi I started an experimental batch of sake yesterday. using bad rice and a tub of koji. we shall see how this goes. today is brew a beer before holidays day to round out the fermenter collection16:48
anteayathere are sandals and hawaiian shirts involved16:48
fungianteaya: dstufft: it's actually a reference to a series of poems from a pulp horror/sci-fi writer from the early 20th century16:48
anteayaclarkb: may you get high on the fumes16:49
*** ociuhandu has joined #openstack-infra16:49
anteayafungi: oooooh16:49
fungiclarkb: chris saw larissa mention it on the faceblocks and told me16:49
anteayaeven more obscure16:49
pleia2re: third party docs, I put an agenda item up for our meeting to determine where we want to land it, seems to be confusion over whether it should go in infra-manual or ci.openstack.org16:49
pleia2so jaypipes knows where to put the patches16:50
*** Hal_ has quit IRC16:50
*** ryanpetrello has quit IRC16:50
*** achanda has quit IRC16:50
*** sputnik13 has quit IRC16:50
*** sputnik13 has joined #openstack-infra16:50
anteayaI'm beginning to question of the name third party is actually the descriptor we want16:52
anteayasince it implies a relationship to openstack and reading the event stream16:52
mtreinishfungi, dstufft: is this pip 6?: http://logs.openstack.org/72/143472/2/check/check-tempest-dsvm-full/5303071/logs/devstacklog.txt.gz#_2014-12-22_16_27_11_12016:52
fungimtreinish: looking16:52
dstufftyes16:52
anteayaand some folks are setting up internal testing systems that replecate our infrastructure but don't interact with it16:53
pleia2anteaya: good point16:53
dhellmannfungi: :-|16:53
dstufftI thought I submitted a CR for that already...16:53
dstufftmaybe I'm high16:53
anteayaI'm finding myself gravitating to the phrase external ci16:53
*** garyk has quit IRC16:53
anteayameaning anything that our infrastruture team is not runnning16:53
dstufftprettytable>0.7 -> prettytable>0.7.0 or prettytable>=0.716:53
anteayawhether it interfaces with our system or not16:54
*** Ala has quit IRC16:54
clarkbjeblair: looks like my gear patch held up16:54
fungidstufft: oh, >0.7 won't work if there's a 0.7.0 and nothing later, because they sort the same. d'oh!16:54
anteayajeblair: interesting article16:54
*** carl_baldwin has quit IRC16:54
anteayamy take away is urea is helpful16:54
dstufftfungi: well not quite16:54
pleia2anteaya: yeah, not even sure "external" is correct, since it's really about taking our open source project that is openstack ci and helping others use it16:55
dstufft> and < are version exclusions operations16:55
dstufftso >0.7 implies !=0.7.*16:55
anteayapleia2: that is true16:55
dstufft>0.7.0 implies !=0.7.0.*16:55
anteayapleia2: as external begs the question, "external to what"16:55
pleia2anteaya: yeah16:55
dstufftfungi: https://bitbucket.org/pypa/setuptools/issue/301/101-in-requirementparse-foo-10-results has some words Iw rote16:56
jeblairclarkb: cool.  i forgot to restart it without logging, but we're okay on space16:56
*** gyee has joined #openstack-infra16:56
*** otter768 has joined #openstack-infra16:57
*** carl_baldwin has joined #openstack-infra16:58
*** avozza is now known as zz_avozza16:58
*** adalbas has joined #openstack-infra16:58
*** spw is now known as sweston16:58
*** sarob has joined #openstack-infra16:59
*** e0ne is now known as e0ne_16:59
fungisorry, was _literally_ herding a cat (had to get it into a crate so it could be taken to boarding before we skip town to see family17:00
pleia2hehe17:00
*** openstack has joined #openstack-infra17:02
jeblairttx: i left a -1 on 139613, do you think you can remove that section?17:02
*** lbragstad has quit IRC17:03
*** pabelanger has quit IRC17:03
*** gondoi has quit IRC17:03
*** roaet_ has quit IRC17:03
*** Masahiro has joined #openstack-infra17:05
anteayafungi: well at least you have one instance of being a successful cat herder17:05
fungiunreported node count in puppetboard is falling rapidly now17:06
*** pabelanger has joined #openstack-infra17:06
*** roaet_ has joined #openstack-infra17:06
*** gondoi has joined #openstack-infra17:07
*** lbragstad has joined #openstack-infra17:07
*** dkranz has quit IRC17:07
anteayaso I'm operating under the belief that project-config/nodepool/scripts are consumed by nodepool, yet I can not grep any mention of install_devstack_dependencies.sh in nodepool: https://review.openstack.org/#/c/142327/3/nodepool/scripts/install_devstack_dependencies.sh17:07
anteayaI could be grepping wrong but I can find prepare_node_devstack.sh17:07
jeblairanteaya: i will answer your question, 1 sec17:08
anteayajeblair: thanks17:08
jeblairanteaya: so over in SYSTEM-config, there is this file: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/nodepool/nodepool.yaml.erb17:09
jeblairanteaya: which is the nodepool config file17:09
fungidstufft: looks like prettytable is already using sane comparisons in all branches for global-requirements.txt so presumably the problem is individual projects which are not in sync with that17:09
dstufftfungi: I think it was a problem with devstack itself17:09
anteayaah sorry yes, fungi showed me this the other day and I forgot17:09
dstufftI could have swore I fixed this, maybe I jsut forgot to submit the CR17:09
dstufftI can't look right this sec though17:10
jeblairanteaya: for each type of nodepool image, it specifies a setup script.  eg: "setup: prepare_node_devstack.sh"17:10
*** Masahiro has quit IRC17:10
jeblairanteaya: that's the connection between nodepool and the scripts in that directory in PROJECT-config17:10
fungidstufft: yeah, i think your memory is correct17:10
*** achanda has joined #openstack-infra17:10
jeblairanteaya: so then generally what happens is that script (or one of the few others like it) will call several of the many scripts in the project-config nodepool scripts dir17:10
ttxjeblair: sure17:11
anteayajeblair: hmmm I can't find the install_devstack_dependencies.sh mentioned in the nodepool config file17:11
*** annegentle has quit IRC17:11
jeblairanteaya: there's no automatic visualization of this, but you can imagine there's a tree structure, where it starts with a script in the nodepool config, and that script calls other scripts which may eventually call the script you are interested in17:11
anteayayes, I can see that, thank you17:12
jeblairanteaya: so what you can do is grep to see what scripts in the nodepool scripts dir call the one you are looking at, and then see which ones call them, and work your way back up the tree until you find one that is mentioned in the nodepool config file17:12
jeblairanteaya: if you do not find such a script, then it really is orphaned and not used :)17:12
openstackgerritThierry Carrez proposed openstack-infra/infra-specs: StoryBoard task branch support  https://review.openstack.org/13961317:12
ttxjeblair: ^17:12
anteayajeblair: awesome, I will grep all the things and report back17:12
dhellmannfungi: so pip 6 has some different behaviors? is there anything I can do to help debug/fix?17:13
*** k4n0 has quit IRC17:13
*** [HeOS] has quit IRC17:13
fungidhellmann: i think it's going to need a devstack patch, as dstufft pointed out17:14
jeblairttx: thx.  krotscheck: can you look at 139613 once more please?17:14
fungidhellmann: working on that now17:14
*** e0ne_ is now known as e0ne17:14
dhellmannfungi: devstack is trying to install prettytable>0.7, which seems fine. Do we know why that's breaking?17:14
dhellmannor is it the deprecated args?17:14
fungidhellmann: prettytable>0.7 means prettytable 0.8 or later. 0.7.1 no longer matches that comparison17:15
clarkbthats lolzy17:15
dhellmannwtf?17:15
clarkbbut ok17:15
* dhellmann misses the days of statically linked C apps17:15
clarkbif its changed to 0.7.0 is that expected to work?17:15
jeblairdhellmann: mordred and i are starting a club.  let me just get your membership packet...17:16
jogothat is od17:16
jesusaurusclarkb: yeah, that should work17:16
jogowhy make changes like this days before holiday? sigh17:17
dhellmannjeblair: Your ideas are intriguing to me and I wish to subscribe to your newsletter.17:17
fungijogo: there's always some holiday somewhere17:17
dhellmannjogo: to encourage us all to take time off?17:17
openstackgerritThierry Carrez proposed openstack-infra/infra-specs: StoryBoard task milestone support  https://review.openstack.org/13962617:18
jogodhellmann: haha17:18
sigmavirus24fungi: along those lines, it's always 5pm somewhere ;)17:18
fungidhellmann: dstufft: https://review.openstack.org/143501 that will probably need backporting to icehouse and juno as well before it will pass grenade, i'm guessing17:18
*** ayoung has quit IRC17:19
fungichances are we'll need to start at icehouse and work upward17:19
*** gyee has quit IRC17:19
*** teran has joined #openstack-infra17:19
*** EmilienM|afk is now known as EmilienM17:19
dhellmannfungi: I guess we no longer get the implicit .0 added so the exclusivity is applied to the full 0.7 range?17:20
fungidhellmann: right17:20
dstufftclarkb: >V (and <V) imply !=V.*. IT's a little wierd for >, but in my experiments > is not used very often compared to >=17:20
*** teran_ has joined #openstack-infra17:20
asselinKatherine, regarding zuul not starting up (in 3rd party meeting conv w/ you and anteaya) it's possible it you have errors in the configuration files.17:20
clarkbya icehouse forward was how we had to do things last time17:20
dstufftit makes more sense when you think about <17:20
dstufft<3 now means "less than 3, and 3.0.dev0 isn't less than 3"17:20
* mtreinish get's his devstack +2 ready17:20
clarkbdstufft it just isnt intuitive. 0.7.2 is > 0.717:20
fungidhellmann: dstufft: 143503 and 143504 are icehouse and juno respectively17:21
*** jpich has quit IRC17:21
fungimtreinish: ^17:21
jesusaurusclarkb: its a precision thing, 0.8 > 0.7 ; 0.7.2 > 0.7.017:21
dhellmannfungi: I only have +1 on devstack, but I voted17:22
mtreinishfungi: I'll go and +A everything when check comes back17:22
asselinKatherine, anteaya I do this to check the files & spit out errors. You can run this command manually outside of puppet: https://github.com/rasselin/os-ext-testing/blob/master/puppet/modules/os_ext_testing/manifests/master.pp#L28717:22
dstufftclarkb: right, the > case isn't super intuitive, but the < case is, and we were choosing between being different in how we treated < and > or doing what a casual reading might expect. Since > wasn't used very often in my estimation of looking through github we chose to be consistent17:22
fungimtreinish: thanks17:22
clarkbjesusaurus except we arent talking scientific measurements. these are softeare versions with expected behavior17:22
dstufftversions aren't numbers so expecting them to act exactly like numbers is going to bite you17:23
*** teran has quit IRC17:24
clarkbthats not the issue. the issue is 0.7.2 > 0.7 has resolved to True always17:24
clarkbyou are right they arent numbers but that relationship isnt controversial its expected17:24
*** sputnik13 has quit IRC17:25
dstufftclarkb: and 3.0.dev0 <3 has resolved to True always, and yet that is an oft complained about thing (and infract, openstack's global requirements has hacky work arounds like <=2.9917:25
clarkbdstufft you can fix that without breaking the above relationship17:26
clarkbits just sort order.17:26
*** armax has quit IRC17:27
*** Sukhdev has joined #openstack-infra17:30
*** dkranz has joined #openstack-infra17:32
anteayaasselin: thank you17:32
asselinanteaya, np17:33
fungipuppetmaster has old nodepool.o.o17:33
dstufftclarkb: can you explain how you'd fix <3 matching 3.0.dev0 without either special casing it or doing what we did17:33
fungi's ssh host key cached17:33
fungifixing17:33
krotscheckttx: Looking17:33
jeblairclarkb, fungi: i'm going to disappear into the bandersnatch hole unless you need me for the next little bit?17:34
SukhdevFolks our CI is hit with this as of this morning - http://paste.openstack.org/show/153936/ - I checked the version of prettytable, it is at 0.7.217:34
fungijeblair: sounds good--i assume you saw the update to your pr over the weekend?17:34
clarkbdstufft use an abstraction like a number like. sort from left to right. a point X to the left of point Y is X<Y17:34
SukhdevAny idea - or has anybody seen this?17:34
jeblairfungi: yep17:35
clarkbdstufft this would then look like: 0.7, 0.7.1, 0.8.0dev0, 0.8, 0.8.117:35
fungiSukhdev: pep 6 was just released and changed what >0.7 means17:35
clarkbgoing left to right as I have writteb them17:35
fungiSukhdev: there are devstack patches submitted to fix it17:35
*** skolekonov has quit IRC17:35
*** gyee has joined #openstack-infra17:36
clarkbthen your comparison is indexof(x) vs indexof(y)17:36
krotscheckjeblair: ttx: Looks good to me.17:36
Sukhdevfungi: do you by any chance has link?17:36
clarkb*number line17:37
anteayajeblair: found the trail the script has a call within project-config, thanks17:37
fungiSukhdev: https://review.openstack.org/#/q/I2134c7d8f58f8b83f33150c9ed86d87f8ccba2f3,n,z17:37
jesusaurusclarkb: how do you ensure that 0dev0 comes before None?17:37
Sukhdevfungi: Thanks17:37
clarkbjesusaurus not sure I follow. 0dev0 > None. but not > 017:38
fungiand yeah, the master branch patch has already failed grenade, so pretty positive we'll have to land icehouse followed by juno first17:38
clarkbNone would likely have to be your logical bottom17:38
*** Ryan_Lane has quit IRC17:38
dstufftclarkb: that's how they sort now, but that doesn't change the fact that 0.8.0dev0 sorts lower than <17:38
dstuffter17:38
dstufftsorts less than 8.017:39
openstackgerritMerged openstack-infra/infra-specs: StoryBoard task branch support  https://review.openstack.org/13961317:39
*** isviridov is now known as isviridov_away17:40
*** mfer has joined #openstack-infra17:40
clarkbdstufft yes 0.8.0dev0 < 0.817:40
zaromorning17:40
*** garyh has joined #openstack-infra17:40
anteayamorning zaro17:40
dstufftclarkb: ok, so how do you ensure that 0.8.dev0 does not match ``<0.8``17:40
*** mfer has quit IRC17:40
clarkbdstufft I dont think you do is what I am saying. instead you dont install prereleases by default which was the old behavior as I understood it17:41
dstufftclarkb: not installing pre-releases by default isn't a fix for that, you shouldn't get 0.8.dev0 if someone did --pre if you have <0.817:42
*** EmilienM is now known as EmilienM|afk17:42
*** koolhead17 has quit IRC17:43
clarkbdstufft maybe the bounday should be on prereleases then not minor versions?17:43
dstufftthe situation is that people expect <V to imply !=V.* but they expect >V to not imply !=V.*, we looked at it, and we found very little use of >V, but we found a good amount of use of <V. So we decided that being consistent in < and > is more important17:44
fungidstufft: is it possible that pip 6 has changed anything around file permissions? http://logs.openstack.org/01/143501/1/check/check-tempest-dsvm-full/c3e41b9/logs/devstacklog.txt.gz#_2014-12-22_17_41_23_53817:45
clarkbthat would require an extra dimension on the line  so you get: 0.7, 0.7.1, (0.8.0dev0, 0.8), 0.8.117:45
*** tnovacik has joined #openstack-infra17:45
* fungi is unstacking thr turtles one by one17:46
clarkbthe base operation for comparison would still be indexof17:46
*** amuller has quit IRC17:46
dstufftfungi: um17:46
*** hemnafk is now known as hemna17:47
clarkbwhich is sort of what the current implementation seems to do it just groups on the wrong boundary imo17:47
dstufftfungi: we did a thing yes, we don't copy over file permissions from the temp dir anymore17:47
*** _nadya_ has quit IRC17:47
fungidstufft: is that a regression or something intentional? just need to know if i need a temporary workaround or a long-term fix (likely to be a whack-a-mole with this situation for some time i expect)17:48
dstufftfungi: not copying over file permissions from the temp dir is an intentional thing, it meant we inhereted permissions and SELinux context from the temporary directory instead of from the destination directory17:50
dstufftthe change means we inherent from the desintation and the fact we had a temp dir is an implementtation detail17:50
dstufftfungi: i'm not sure if that's what's causing your error though17:51
sandywalshanything going on with review.openstack.org? Unable to send new branch.17:51
hemnahow does one run the javelin tests standalone ?17:52
anteayasandywalsh: well rax is having dns issues, might that be a factor?17:52
fungidstufft: so only directory permissions not per-file permissions then?17:52
sandywalshanteaya: hmm, could be ... thanks17:52
anteayasandywalsh: np17:53
openstackgerritMerged stackforge/python-jenkins: Fix enable_node and disable_node methods  https://review.openstack.org/13895417:53
*** pcaruana has quit IRC17:53
tnovaciki got server not found on https://review.openstack.org/ - is site down?17:54
fungitnovacik: our dns provider is having trouble. see https://status.rackspace.com/17:55
anteayahemna: javelin is part of grenade (this is the sum total of my javelin knowledge)17:55
Sukhdevfungi: I tested your patch - it works. It fixed the issue for me - thanks17:56
anteayahemna: perhaps some grenade folks in -qa might be of more assistance17:56
tnovacikfungi, thx17:56
*** derekh has quit IRC17:56
fungiSukhdev: there are other errors with it now too, so that's probably just the first step to getting devstack/tempest working with pip 617:56
*** annegentle has joined #openstack-infra17:57
hemnaanteaya, that's kinda what I figured.  just trying to figure out how to run em.17:58
*** annegentle has quit IRC17:58
Sukhdevfungi: I have been upgrading lots of packages to get going….I was stuck on this one….this seems to have taken care of my issue - until I hit the next one :-):-)17:59
anteayahemna: I hear ya17:59
*** annegentle has joined #openstack-infra18:00
mtreinishhemna: javelin is part of the tempest repo: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/cmd/javelin.py18:00
mtreinishhemna: if you need help with it we can move over to -qa and discuss it there18:00
hemnamtreinish, yah I'm looking at it now, trying to track down a failure with one of my patches.18:01
mtreinishhemna: link?18:01
*** ryanpetrello has joined #openstack-infra18:01
*** dims has quit IRC18:01
*** mfer has joined #openstack-infra18:02
*** ddieterly has joined #openstack-infra18:02
*** dims has joined #openstack-infra18:02
*** BobBall is now known as BobBall_Xmas18:02
*** patrickeast has joined #openstack-infra18:02
dtroyer+A on the prettytable reviews…I thought we removed that whole section recently though…still looking18:03
*** ChuckC has quit IRC18:03
*** cnesa has quit IRC18:04
*** redixin has quit IRC18:04
*** sarob has quit IRC18:04
fungidtroyer: do you know enough about django to tell me what provides /usr/local/bin/django-admin.py ?18:04
anteayamtreinish hemna so even my miniscule knowledge of javelin is wrong18:05
dtroyerI know django is spelled with a silent 'd'18:05
fungidtroyer: awesome. that makes two of us!18:05
*** ddieterl_ has quit IRC18:05
dstufftfungi: django-admin.py is I think a distutils script18:06
*** cnesa has joined #openstack-infra18:06
mtreinishanteaya: it get's run as part of grenade but the code lives in the tempest repo (just to reuse the tempest api clients)18:06
dstufftfrom the Django package18:06
anteayamtreinish: thank you18:06
fungidstufft: thanks. that narrows it down at least18:06
*** dims has quit IRC18:06
mtreinishanteaya: there is a patch up adding more docs for it: http://docs-draft.openstack.org/23/138423/3/check/gate-tempest-docs/ed7a8c9//doc/build/html/javelin.html18:06
*** bdpayne has joined #openstack-infra18:07
anteayathanks, also I tracked you to -qa18:07
*** bradjones has quit IRC18:07
*** hichtakk has joined #openstack-infra18:08
fungiyeah, looks like _something_ has changed in the past hour or so to make devstack master start failing on the call to django-admin.py. i can imagine an easy fix to that, but trying first to figure out what brought it on (whether pip 6 or something else unrelated)18:09
fungilooks like the stable branches don't call that, so it may only need addressing in master at least18:10
*** cnesa has quit IRC18:10
openstackgerritElizabeth K. Joseph proposed openstack-infra/system-config: Add Wildfly Puppet module  https://review.openstack.org/14351218:11
*** bradjones has joined #openstack-infra18:12
fungijenkins01 is a might annoyed18:12
fungivery slow webui18:12
fungitook a while to catch up with puppet reporting compared to its siblings18:13
*** wuhg has quit IRC18:13
*** yolanda has quit IRC18:14
*** yolanda has joined #openstack-infra18:15
fungijobs running from it are taking longer than others too, especially for things which happen master-side like publishers18:15
jogoheyongli: http://192.55.68.190/141719/1/ looks pretty good18:15
asselinCan we get these 3 module-split spec patches approved & merged? https://review.openstack.org/#/q/topic:module-split+status:open+project:openstack-infra/infra-specs,n,z18:15
jogoheyongli: one thing missing is a link to where the tests can be found18:16
*** pblaho_ has quit IRC18:16
dstufftfungi: sorry I'm still doing some things with the release, can you see if you can reproduce the Django thing outside of devstack?18:16
jogoheyongli: as for commenting on nova patches, lets talk about this either on the openstack ML or in the next nova meeting (after new years)18:17
fungidstufft: that's what i'm presently doing18:19
fungidstufft: juggling that with the other fixes already in progress18:19
*** SumitNaiksatam has quit IRC18:19
dstufftfungi: no problem, i'm juggling too heh (on top of the pip release stuff, My wife is wrapping presents which means I'm getting sent into the basement to locate more gifts that need wrapped periodically :V)18:19
fungidstufft: amusingly, same here ;)18:20
*** ryanpetrello has quit IRC18:24
fungidstufft: for a second data point, ironic seems to be failing on "18:25
*** bswartz has quit IRC18:25
fungi/usr/local/bin/ramdisk-image-create: Permission denied"18:25
fungihttp://logs.openstack.org/04/143504/1/check/check-tempest-dsvm-ironic-pxe_ssh/ec4173c/logs/devstacklog.txt.gz#_2014-12-22_17_49_41_34818:25
*** bswartz has joined #openstack-infra18:26
openstackgerritElizabeth K. Joseph proposed openstack-infra/puppet-zanata: Add Rakefile, TODO & Wildfly configuration support  https://review.openstack.org/14264518:27
fungiand unlike the django-admin.py issue, that one's also affecting stable/juno devstack changes so we'll need to solve it before we can get juno passing the gate18:28
*** jedimike has quit IRC18:31
*** hichtakk has quit IRC18:32
fungidstufft: confirmed it's pip 6.0 doing it... http://paste.openstack.org/show/153945/ now going through the pip logs for signs of why18:32
dstufftfungi: can you try without --no-use-wheel18:32
dstuffter with18:32
fungiyep!18:32
*** armax has joined #openstack-infra18:35
dstufftI suspect I might know what it is18:35
fungidstufft: yep, that solves it18:35
*** ChuckC has joined #openstack-infra18:35
fungidstufft: when i run with --no-use-wheel i also see:18:35
fungichanging mode of build/scripts-2.7/django-admin.py from 644 to 75518:35
fungichanging mode of /home/fungi/work/openstack/scratch/django-pip6.0/bin/django-admin.py to 75518:35
*** thedodd has quit IRC18:35
fungidstufft: looks like django's setup.py uses scripts=['django/bin/django-admin.py']18:37
*** garyk has joined #openstack-infra18:37
fungito declare it18:38
*** markmcclain has quit IRC18:39
*** dims has joined #openstack-infra18:40
fungilooks like we're about 5 minutes from un-breaking icehouse, but we'll need to figure out the right way forward with these script executable permissions before we can fix juno and ultimately master18:40
dstufftfungi: moment18:40
ZZelleHi everyone!18:40
*** stevemar has quit IRC18:41
ZZelleIt seems some jobs fail to install prettytable with pip18:43
*** salv-orlando has quit IRC18:43
fungiZZelle: yeah, we're working fixes through the gate18:43
fungiZZelle: https://review.openstack.org/#/q/I2134c7d8f58f8b83f33150c9ed86d87f8ccba2f3,n,z18:43
fungiZZelle: because of grenade we have to fix icehouse first, then juno, then master18:43
fungiZZelle: it's a version matching behavior change in pip 6.0, just released an hour ago18:44
morganfainbergoh fun.18:44
ZZellefungi, outch18:44
ZZellefungi, fast healing as usual :)18:44
fungiif this is a time of year you celebrate some particular holiday, i'd suggest doing that instead for a little while ;)18:45
morganfainbergfungi, is this going to impact my ability to release keystonemiddleware (minor patch to unbreak zaquar gate)? because i can hold off if needed.18:45
*** Katherine has quit IRC18:45
fungimorganfainberg: releasing should work, it's just devstack which is presently broken18:45
morganfainbergfungi, good to know.18:46
morganfainbergfungi, i'll think about this because if we have something inadvertently broken in the package i want to be able to push a fix through - we might want to wait for release in case we need fixes in the immidiate term18:47
fungimorganfainberg: well, i _say_ that, but ultimately that's just the only broken i'm aware of so far. you could certainly end up finding some new broken for us18:47
morganfainbergnot that i expect keystonemiddleware is broken18:47
morganfainbergfungi, hehe18:47
*** sc68cal_ is now known as sc68cal18:47
dstufftfungi: https://github.com/pypa/pip/pull/224818:48
sigmavirus24morganfainberg: you're not doing your job if you aren't confident the software you're releasing is broken in some subtle way18:48
dstufftfungi: can you test that18:48
morganfainbergsigmavirus24, haha18:48
morganfainbergsigmavirus24, it's the less subtle but unchecked ways i worry about18:48
fungidstufft: thanks, doing now18:48
dstufftif that works I need make sure it doesn't blow up on Windows18:49
sigmavirus24morganfainberg: all I'm hearing is that you need to write more tests =P18:49
morganfainbergsigmavirus24, well that i *really* worry about18:49
morganfainbergsigmavirus24, haha, we do! but uhm... what test is missing? ;)18:49
morganfainbergsigmavirus24, ^_^18:49
sigmavirus24the test you don't know is missing is the test that is missing18:49
dstufftif you really want the fun of releasing subtly broken software, come work on packaging!18:49
sigmavirus24(I got that one out of a fortune cookie)18:50
*** harlowja has joined #openstack-infra18:50
*** [HeOS] has joined #openstack-infra18:50
*** e0ne has quit IRC18:51
*** achanda has quit IRC18:51
fungidstufft: yeah, did you _see_ how many pbr releases i made over the weekend? :/18:51
*** ociuhandu has quit IRC18:51
dstufftfungi: if it's less than 8 you're still doing better than we did with setuptools + PEP 44018:51
*** achanda has joined #openstack-infra18:51
morganfainbergsigmavirus24, haha18:51
*** thedodd has joined #openstack-infra18:52
fungidstufft: it's coming close. we're now up to 7 pbr point releases in the wake of setuptools 818:52
morganfainbergdstufft, i am scared by packaging - no really, it's a scary place with lots of dark corners, there may in-fact be a grue hiding in there18:52
*** SumitNaiksatam has joined #openstack-infra18:52
fungimorganfainberg: i have already been eaten by a grue18:53
dstufftmorganfainberg: a grue might explain why so many people go missing when they offer to help and we show them the setuptools/pip code base18:53
morganfainbergfungi, damn.18:53
sigmavirus24dstufft: a grue ate my betamax branch for pip's tests18:53
morganfainbergdstufft, i suggest handing them a torch while you're at it then!18:53
sigmavirus24morganfainberg: I received one. it went out shortly after18:53
jaypipesjeblair, clarkb: so... I was able to determine a pattern finally on that 401 Unauthorized thing in gertty...18:53
*** sputnik13 has joined #openstack-infra18:53
*** Masahiro has joined #openstack-infra18:54
jaypipesjeblair, clarkb: turns out that *exactly* one hour after I start gertty, the next HTTP API call to review.openstack.org results in a 401 Unauthorized. very odd.18:54
*** achanda has quit IRC18:54
fungidstufft: the pip/wheel.py file you're modifying in that pr doesn't look like the lib/python2.7/site-packages/pip/commands/wheel.py i'm getting when i install pip 6.018:54
*** achanda_ has joined #openstack-infra18:54
dstufftfungi: wat18:55
dstufftfungi: what does it look like18:55
fungidstufft: after some imports it starts out defining a WheelCommand class18:56
sigmavirus24fungi: I think the difference is that one is used as a library by other commands and one defines a command18:56
dstufftoh wait18:56
jaypipesjeblair: http://paste.openstack.org/show/153947/ <-- the log from gertty for the time period. you'll notice a few extra log messages, which are just in my local version of gertty.18:56
dstufftpip/commands/wheel.py vs pip/wheel.py18:56
dstufftbecause you know, this isn't confusing at all18:56
fungidstufft: just trying to figure out how to try out that patch18:56
sigmavirus24fungi: https://github.com/pypa/pip/pull/2248.patch and apply18:57
jaypipesjeblair: sorry, got cut off log on last link... here's the updated one: http://paste.openstack.org/show/153948/18:58
*** sputnik13 has quit IRC18:58
*** otter768 has joined #openstack-infra18:58
jaypipesjeblair: :( still cut off...18:58
*** Masahiro has quit IRC18:58
dstufftfungi: no I mean there are two wheel.py files in pip18:58
dstufftpip/wheel.py and pip/commands.wheel.py18:58
dstufftmy patch is against the first one18:58
*** jedimike has joined #openstack-infra18:58
dstufftyou're looking at the second18:58
fungisigmavirus24: yeah, i can patch that into the source. i guess i need to reinstall pip from source rather than trying to hand-patch the installed file18:58
jaypipesfungi: is there a paste.openstack.org past limit?18:59
jaypipespaste limit...18:59
*** sputnik13 has joined #openstack-infra18:59
fungijaypipes: i think so, don't remember18:59
fungijaypipes: i usually have trouble past about 1000 lines18:59
fungidstufft: d'oh, right you are19:00
fungidstufft: i missed that my find returned three wheel.py files. sorry!19:00
*** jedimike has quit IRC19:01
jaypipesjeblair: OK, snipped and finally shows the traceback and timing of the 401... sorry for the noise earlier: http://paste.openstack.org/show/153949/19:01
fungidstufft: that does definitely seem to fix it19:02
fungidstufft: pip 6.0 with that patch applied, installing django into a virtualenv without --no-use-wheel, i get -rwxr-xr-x perms on django-admin.py now19:03
jeblairjaypipes: what timezone is that?19:03
jaypipesjeblair: EST19:03
*** otter768 has quit IRC19:04
dstufftfungi: ok19:04
*** sputnik13 has quit IRC19:04
*** Ryan_Lane has joined #openstack-infra19:04
jaypipesjeblair: think it has something to do with daylight savings or something?19:04
dstufftfungi: I'm going to release a 6.0.1 soon19:05
jeblairjaypipes: no, just wanted to correlate with gerrit log19:05
dstufftonce tests pass and stuff19:05
jaypipesjeblair: k19:05
*** sputnik13 has joined #openstack-infra19:05
dstufftreleasing things is the best RC19:05
fungidstufft: great--i'll keep tabs on our mirrors and recheck our other fixes which are blocked on this as soon as it shows up--thanks again!19:05
*** otter768 has joined #openstack-infra19:06
dstufftI hate that there is like 4 different ways to do everything19:06
dstufftin packaging19:06
*** _buzz has quit IRC19:06
dstufftI'm slowly trying to reduce that19:06
dstufft:/19:06
fungidstufft: i wish we had a better way to test unreleased source of things like this without breaking everything, but unfortunately the potential interactions are so numerous there's no easy and direct way to test something like that without just using it everywhere and seeing what breaks19:06
openstackgerritElizabeth K. Joseph proposed openstack-infra/project-config: Add more publications branches to gerritbot  https://review.openstack.org/14319719:07
reedwhat's the procedure to merge duplicated gerrit accounts?19:07
dstufftfungi: we used to release RCs for pip and virtualenv, but nobody ever really tested them19:07
dstufftso we stopped doing that for 6.019:07
tnovacikcould someone point me to location where i can find how are gates configured? (openstack instance on them - what services are installed by default)19:07
fungidstufft: honestly the workflow for us would have been the same regardless19:08
fungireed: in gerrit?19:08
dstufftfungi: yea19:08
reedfungi, yes, there is a duplicate bkobagan@redhat.com19:08
fungidstufft: either switch to release candidates early and solve all the breakage, or wait for it to release and solve all the breakage19:08
reedhe wrote to -infra mlist19:08
dstufftpip 6.0 is going to make PyPI's download counts plummet19:08
dstufftyay on by default HTTP caching and PyPI serving a cache-Control of a year or so19:09
fungireed: there is no feature for that. we just disable one or more gerrit accounts, or else make modifications to various interrelated database tables in gerrit's backend19:09
anteayareed: I saw the ml post but fungi was too busy to bug about it19:09
reedhe asked to delete his accounts and he'll start over, not sure that's an option19:09
anteayareed: since fungi needs to modify the gerrit db by hand19:09
anteayareed: well it is an option but not the best19:10
reedwow19:10
*** ayoung has joined #openstack-infra19:10
anteayareed: yeah, which is why I didn't bring it up today19:10
fungireed: we can mark his existing accounts inactive and let him start over19:10
reedok, I'll leave it to you. He emailed me too so that's why I came here19:10
fungireed: thanks for the heads up19:10
crinkleclarkb: nibalizer fungi we just made a new release of the rabbitmq module19:10
reedthanks19:10
anteayareed: right well since he managed to create 4 accounts, it would also be helpful to find out what he is doing19:11
anteayareed: otherwise he likely will keep doing it19:11
fungicrinkle: thanks for seeing that through!19:11
asselinanteaya, can you +2 these patches? We should get them merged too. https://review.openstack.org/#/q/topic:module-split+status:open+project:openstack-infra/infra-specs,n,z19:11
jeblairjaypipes: i don't see anything useful in gerrit logs.  i'm not up to speed on the problem though, and i'm heads down in something else right now; can you make sure there's a story for it?19:12
*** zz_sabarij is now known as zz_zz_sabarij19:12
anteayaI don't have +2 on infra-spec, just project-config19:12
anteayaasselin: I can review them but can't merge them19:12
jaypipesjeblair: there's a story for it already. no problem, though, you've got better things to do! :) I'll keep digging.19:12
asselinanteaya, ok, I'll gladly take your +1 :) then go to the cores19:13
jeblairjaypipes: heh, not sure about better, but different and equally puzzling.19:13
*** _nadya_ has joined #openstack-infra19:13
jaypipesjeblair: :)19:13
jaypipesjeblair: FYR: https://storyboard.openstack.org/#!/story/200008419:13
*** hdd has quit IRC19:14
jaypipesfungi, jeblair: has anyone reported failures about "No distributions matching the version for prettytable>0.7" yet?19:15
fungijaypipes: yep, https://review.openstack.org/#/q/I2134c7d8f58f8b83f33150c9ed86d87f8ccba2f3,n,z should fix19:16
jaypipesfungi: right on. thx.19:16
* dims was just about to ask :)19:18
anteayathe next lineup19:18
dimsfungi: which review is for "script permission issue with pip 6.0 and wheels is solved"? just curious19:20
dstufftthat's a pip side issue19:21
dstuffthttps://github.com/pypa/pip/pull/224819:21
fungidims: pip 6.0.1/virtualenv 12.0.1 presumably19:21
dstufftI need to spin up a windows machine to make sure it works there19:21
dstufftunless someone has a windows machine who wants to test it :D19:21
fungidstufft: wish i could help, but i would break out in hives. sorry you have to don the hazmat suit19:22
dimshehe,19:22
openstackgerritMichael Krotscheck proposed openstack-infra/infra-specs: Storyboard Task Metadata  https://review.openstack.org/14351719:23
dstufftand yea, 6.0.1 and 12.0.119:23
*** stevemar has joined #openstack-infra19:23
dstufftI'm running tests on all the changes I want in, and spinning up the windows machine now19:24
krotscheckzaro: ^^19:24
*** dprince has quit IRC19:29
*** cody-somerville has joined #openstack-infra19:29
*** cody-somerville has quit IRC19:30
*** cody-somerville has joined #openstack-infra19:30
*** Sukhdev has quit IRC19:30
*** reed has quit IRC19:31
jeblairkrotscheck: i took zaro's question as implying the need for individual addressibility of tasks, so that commits could specify that they addressed a task, rather than a story19:32
*** teran_ has quit IRC19:33
jeblairkrotscheck: i think it brussels we were thinking that tasks might be addressed by a unique string.  so "Story: 1234-abcd" is the id for a story-task, and that's what we would put in commit messages"19:34
jeblairkrotscheck: i think it brussels we were thinking that tasks might be addressed by a unique string.  so "1234-abcd" is the id for a story-task, and that's what we would put in commit messages" (revised)19:35
*** hichtakk has joined #openstack-infra19:35
*** sputnik13 has quit IRC19:36
krotscheckjeblair: Something like this? https://storyboard.openstack.org/#!/story/32619:36
*** sputnik13 has joined #openstack-infra19:37
* anteaya goes for a walk19:37
jeblairkrotscheck: yeah that one.  i'll fill in with some quick notes from the conversation in bru19:38
fungidstufft: i'm suspecting that the zuul merge-check failure on https://review.openstack.org/142593 indicates a race where it was queued for tested against the branch tip but then merged before it actually got tested, and so merge-conflicted with itself essentially19:38
fungidstufft: er, sorry19:38
krotscheckkk19:38
fungidhellmann: ^19:38
dhellmannfungi: does that mean it merged properly?19:39
fungidhellmann: yes, it merged properly19:39
dhellmannfungi: ok, cool19:39
fungiit's in the branch (and now released even)19:40
*** garyk has quit IRC19:40
*** lttrl has joined #openstack-infra19:42
greghaynesfungi: Hey, if everything is not currently on fire clarkb mentioned you might have a good grasp on an issue were running into with pbr: https://bugs.launchpad.net/tripleo/+bug/140494819:43
uvirtbotLaunchpad bug 1404948 in tripleo "Tests failing with Versioning for this project requires either an sdist tarball, or access to an upstream git repository" [Critical,Confirmed]19:43
*** salv-orlando has joined #openstack-infra19:43
greghaynes(so you can blame him for throwing you under the bus ;)19:44
fungigreghaynes: you confirm that's happening even with pbr 0.10.7? can you try with https://review.openstack.org/142841 and its dependency on master?19:45
greghaynespbr-0.10.7-py2.py3-none-any.whl is the package that gets installed. /me tries with that patch19:46
*** mfer has quit IRC19:48
*** mjturek has quit IRC19:48
*** sputnik13 has quit IRC19:48
*** salv-orlando has quit IRC19:48
greghaynesfungi: installing pbr with that patch beforehand seems to work19:50
fungigreghaynes: preinstalling pbr 0.10.7 and then installing python-cinderclient from source as "." seems to work19:50
fungistill working on reproducing19:50
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration to add test attrs column to test_runs table  https://review.openstack.org/14323919:51
greghaynesfungi: huh. so we install from abs path... wonder if thats it19:51
*** annegentle has quit IRC19:52
fungigreghaynes: i'm trying that now19:53
*** markmcclain has joined #openstack-infra19:53
greghaynesfungi: wow, so preinstalling with pbr==0.10.7 doesnt cause the issue19:54
greghaynesbut with pbr>=0.5.21,<1.0 does19:54
greghaynesso seems like a requirements parsing issue19:55
*** e0ne has joined #openstack-infra19:56
*** e0ne is now known as e0ne_19:56
fungigreghaynes: can you try to reproduce with virtualenv earlier than 12.0?19:56
*** e0ne_ is now known as e0ne19:56
greghaynesI am using 1.11.4 currently19:56
fungik19:56
*** otter768 has quit IRC19:58
*** mjturek has joined #openstack-infra19:58
greghaynesalso, it appears that pbr doesnt show up in pip freeze19:58
*** mjturek has left #openstack-infra19:59
greghaynesoh, nvm, that was me being a derp and running the wrong pip19:59
fungigreghaynes: if it really is newer pbr, then you should also have a `pbr freeze` command which will list git shas for you too20:00
greghaynesI dont think its newer pbr - this issue just started this morning20:00
greghaynesso best guess is its new pip related20:00
fungigreghaynes: what version of pip do you have showing up in that virtualenv when you successfully reproduce the error?20:01
*** [HeOS] is now known as HeOS20:01
greghaynespip 6.020:01
jeblairfungi, mordred: bandersnatch pr updated20:02
fungigreghaynes: if it's just been happening in the past few hours, then yes that probably coincides20:02
fungigreghaynes: we likely have new pbr bugs in the wake of the pip 6.0 release, but right now can't easily test it or land patches because we're blocked on a pip bug in devstack, so i'll attempt to unravel it further for now but likely won't have good progress for a while20:03
fungigreghaynes: i confirm that updating the venv to pip 6.0 i'm able to reproduce your error at least20:03
greghaynesfun fun20:03
fungigreghaynes: looking now to see if i can pin down the cause20:04
greghaynesok, thats good then :)20:04
dstufftfungi: https://github.com/pypa/pip/pull/225420:04
*** rfolco has quit IRC20:04
dstufftonce tests pass I'll release 6.0.120:04
fungidstufft: those all look like excellent improvements. thanks!20:04
*** Longgeek has quit IRC20:04
openstackgerritElizabeth K. Joseph proposed openstack-infra/puppet-zanata: Add Rakefile, TODO & Wildfly configuration support  https://review.openstack.org/14264520:05
*** hichtakk has quit IRC20:07
*** hichtakk has joined #openstack-infra20:07
*** zz_zz_sabarij is now known as zz_zz_zz_sabarij20:08
asselinfungi, jeblair can we get these merged? https://review.openstack.org/#/q/topic:module-split+status:open+project:openstack-infra/infra-specs,n,z20:09
fungigreghaynes: marvellous! i can't even self-install pbr from source with pip 6.020:11
greghayneshaha, much better repo case20:11
fungigreghaynes: `pip install .` breaks with the same error20:11
jeblairasselin: done20:13
*** annegentle has joined #openstack-infra20:13
openstackgerritMerged openstack-infra/infra-specs: Update module issues_url & include storyboard  https://review.openstack.org/13545220:13
openstackgerritMerged openstack-infra/infra-specs: Remove 2 modules that no longer exist  https://review.openstack.org/14262720:14
fungigreghaynes: i've tested back as far as 0.8.0 and get the same behavior20:15
openstackgerritMerged openstack-infra/infra-specs: Use the correct project name  https://review.openstack.org/14292020:15
openstackgerritSumit Naiksatam proposed openstack-infra/project-config: Update GBP projects for tarballs and gerritbot  https://review.openstack.org/14352620:16
*** hichtakk has quit IRC20:17
*** annegentle has quit IRC20:21
openstackgerritSumit Naiksatam proposed openstack-infra/project-config: Update GBP projects for tarballs, pypi & gerritbot  https://review.openstack.org/14352620:22
*** hichtakk has joined #openstack-infra20:25
*** mnaser has joined #openstack-infra20:31
mnaserwhere's the best place to report a typo in http://developer.openstack.org/api-ref-networking-v2.html ?20:31
fungidstufft: does pip 6.0 possibly change its operating path, compared to 1.5.6? i've boiled our pbr bug down to a subprocess call for 'git rev-parse --git-dir' in the cwd exiting 128 (fatal: Not a git repository)20:33
*** EmilienM|afk is now known as EmilienM20:34
*** tnovacik has quit IRC20:34
sc68calmnaser: I think https://bugs.launchpad.net/openstack-manuals20:34
dstufftfungi: it's possible, nothing is standing out in my mind as far as what chnages I can think of20:34
*** sputnik13 has joined #openstack-infra20:35
mnaserthanks sc68cal20:35
*** sputnik13 has quit IRC20:39
openstackgerritSumit Naiksatam proposed openstack-infra/project-config: Update GBP projects for tarballs, pypi & gerritbot  https://review.openstack.org/14352620:42
*** Masahiro has joined #openstack-infra20:42
*** annegentle has joined #openstack-infra20:43
*** annegentle has quit IRC20:44
*** Guest32161 is now known as annegentle20:44
*** ryanpetrello has joined #openstack-infra20:44
fungidstufft: it looks like the pip build dir for pbr, when using pip 1.5.6, is a valid git repository... but under 6.0 it isn't. i'm still trying to unwrap enough of pbr to figure out how it gets that into the pip build dir20:47
*** Masahiro has quit IRC20:47
*** mnaser has left #openstack-infra20:48
*** salv-orlando has joined #openstack-infra20:48
*** enikanorov_ has joined #openstack-infra20:49
*** enikanorov has quit IRC20:51
*** salv-orlando has quit IRC20:51
*** dkranz has quit IRC20:52
dstufftfungi: does it ever use the pip.locations._get_build_prefix() function20:53
openstackgerritMerged openstack-infra/infra-manual: Add .gitreview content to Creator's Guide  https://review.openstack.org/13883120:53
fungidstufft: git grep says no occurrences of the substrings get_build_prefix or location in the pbr source tree20:54
dstufftfungi: what about pip_build20:54
fungiexcept for some local variable names anyway20:54
fungichecking20:54
funginope, not that either20:54
sigmavirus24jogo: of probable interest to you https://gitlab.com/pycqa/flake8/issues/1720:56
jogoohh fun :/20:57
*** harlowja_ has joined #openstack-infra20:58
*** harlowja has quit IRC21:00
fungimordred: clarkb: dhellmann: any hints on how pbr normally gets the git repository into the pip tmp build dir when installing from source?21:00
openstackgerritJulien Danjou proposed openstack-infra/git-review: Allow to specify which commit to send  https://review.openstack.org/13970121:00
fungimordred: clarkb: dhellmann: or is it relying on pip to copy the contents of the directory tree for it?21:01
openstackgerritMerged openstack-infra/infra-manual: Expand more of the repo verification steps  https://review.openstack.org/14082821:01
dstufftoh21:01
dstuffthaa21:01
dstufftI know what it is21:01
fungi?21:01
fungidid pip 6.0 start skipping dotfile/dotdirs when it copies?21:01
* fungi is grasping at straws now21:02
*** unicell has quit IRC21:02
dstuffthttps://github.com/pypa/pip/pull/219621:02
dstufftwe should probably just revert that in 6.0.121:03
dstufftor drop the vcs repos from them21:03
fungidstufft: d'oh!21:03
fungidstufft: that _is_ it21:03
*** teran has joined #openstack-infra21:03
dstufftI gotta go pick up my daughter21:03
dstufftI'll fix that as aprt of 6.0.1 too when I get back21:03
fungidstufft: thanks a bunch!21:04
*** achanda_ has quit IRC21:04
*** achanda has joined #openstack-infra21:05
*** dkranz has joined #openstack-infra21:07
greghayneswow21:07
greghayneswelp, that would do it :)21:07
*** teran_ has joined #openstack-infra21:07
*** ryanpetrello has quit IRC21:08
fungigreghaynes: so anyway, should hopefully be resolved when pip 6.0.1 fixes the world21:08
fungiin the meantime, i need to wrap some gifts and pack luggage21:08
greghaynesyep, ty for the help!21:08
fungigreghaynes: thank dstufft. he was the one who figured it out21:08
*** salv-orlando has joined #openstack-infra21:09
greghaynestyty!21:09
*** teran has quit IRC21:09
fungii just bumbled around sticking debug calls into the pbr source tree and generally being useless21:09
*** andreykurilin_ has joined #openstack-infra21:09
*** ociuhandu has joined #openstack-infra21:09
*** achanda has quit IRC21:09
anteayafungi: when are you off?21:10
*** Ryan_Lane has quit IRC21:11
fungianteaya: always, but i have varying degrees of internets when on planes, in cars, visiting family, whatever21:11
*** hichtakk has quit IRC21:11
anteayaha ha ha21:11
*** hichtakk has joined #openstack-infra21:11
anteayathe packing of the luggage21:11
fungiyeah, road trip for a good chunk of tomorrow21:11
anteayaah cool, may you have safe travels21:12
mattoliverauMorning all21:13
anteayamorning mattoliverau21:14
fungi"foo-X.Y.Z does not satisfy foo>X.Y" has now blown up as a thread on the distutils-sig ml. whee...21:15
lifelessof course21:15
lifelessbecause its different to how ever other version relation in software works21:15
*** dims has quit IRC21:20
*** zz_zz_zz_sabarij is now known as zz_zz_zz_zz_saba21:21
*** dims has joined #openstack-infra21:21
*** dims has quit IRC21:23
*** dims has joined #openstack-infra21:23
*** teran has joined #openstack-infra21:23
jheskethMorning21:25
dimsfungi: safe travels21:25
*** signed8bit has joined #openstack-infra21:25
fungithanks dims21:25
anteayamorning jhesketh21:26
*** teran_ has quit IRC21:26
*** kgiusti has quit IRC21:31
*** ddieterly has quit IRC21:32
*** _nadya_ has quit IRC21:33
*** ddieterly has joined #openstack-infra21:34
*** adriant has joined #openstack-infra21:35
adriantany gerrit administrator about who can add me to a newly added stackforge project?21:37
*** prad has joined #openstack-infra21:37
anteayaadriant: hello21:37
anteayahave you got a patch up to project-config?21:38
anteayaor are you asking how to create the patch?21:38
adriantproject is merged, how do I add myself as a core/release for it?21:38
anteayaah21:38
anteayawhich project?21:38
adriantstackforge/cloud-pydashie21:38
anteayafungi: ^21:39
*** teran_ has joined #openstack-infra21:39
stevemarwondering why i'm getting this error, apparently prettytable is not available at the pypi host http://pypi.dfw.openstack.org -> http://logs.openstack.org/13/143513/1/check/check-osc-dsvm-functional/d73a071/logs/devstacklog.txt.gz21:39
anteayastevemar: this is an issue with pip6.021:40
anteayaand fungi has some patches up21:40
anteayalast I looked one had merged three to go21:40
nibalizertttttntnttn21:42
nibalizeroops21:42
*** dkranz has quit IRC21:42
anteayainteresting21:42
fungistevemar: anteaya: yes, that specific error will be addressed once all of https://review.openstack.org/#/q/I2134c7d8f58f8b83f33150c9ed86d87f8ccba2f3,n,z merge, but we're blocked on an updated pip/virtualenv point release to address some freshly-discovered regressions there21:42
*** teran has quit IRC21:42
*** pblaho_ has joined #openstack-infra21:42
*** isq has joined #openstack-infra21:42
anteayaoh awesome21:44
fungiadriant i've added you to the cloud-pydashie-core and cloud-pydashie-release groups in gerrit now21:44
anteayaand here I thought the fun was almost over for Monday21:44
*** tonytan4ever has quit IRC21:44
anteayawell from my bystander's point of view21:44
adriantfungi: thanks, just saw :)21:44
fungianteaya: hardly. once we get devstack patched up, we've still got pbr changes to land to un-break its master branch21:44
*** baoli has quit IRC21:45
pleia2nibalizer: haha, you gave yourself away! I would appreciate your feedback on this when you have a chance https://review.openstack.org/#/c/142645/21:45
*** signed8bit has quit IRC21:45
anteayafungi: ah see, I missed all of that in my trying to keep up21:46
fungipleia2: it was just his cat walking across the keyboard. the cat also typed "oops" afterward21:46
anteayapleia2: try nibalizer's cat21:46
pleia2:)21:46
nibalizerpleia2: sure thing21:46
nibalizerno what happens is ssh lags and then i spam characters21:46
stevemarfungi, ah okay... thanks for the heads up as usual21:47
pleia2nibalizer: thanks \o/21:48
*** ChuckC has quit IRC21:48
*** melwitt has joined #openstack-infra21:49
*** teran has joined #openstack-infra21:50
*** hichtakk has quit IRC21:53
*** hichtakk has joined #openstack-infra21:53
*** teran_ has quit IRC21:53
*** andreykurilin_ has quit IRC21:53
*** sdake_pto has quit IRC21:54
*** dkranz has joined #openstack-infra21:54
*** sdake has joined #openstack-infra21:54
*** sdake has quit IRC21:54
*** sdake has joined #openstack-infra21:54
*** teran_ has joined #openstack-infra21:55
*** teran has quit IRC21:57
*** e0ne has quit IRC21:58
*** otter768 has joined #openstack-infra21:59
*** dkliban is now known as dkliban_afk21:59
*** bswartz has quit IRC22:01
dstufftfungi: https://github.com/pypa/pip/pull/225722:01
*** otter768 has quit IRC22:04
*** pc_m has quit IRC22:07
*** dkranz has quit IRC22:09
anteayawow an almost empty zuul22:10
*** signed8bit has joined #openstack-infra22:11
mtreinishanteaya: it's because dsvm jobs get thrown out early because of the pip stuff22:13
mtreinishcombined with people taking vacation :)22:13
anteayamtreinish: we should consider that a feature with a toggle button22:14
*** e0ne has joined #openstack-infra22:14
anteayaand yay holidays22:15
*** pblaho_ has quit IRC22:15
*** spzala has joined #openstack-infra22:16
bnemecslagle-afk: Just a heads up, I can't build os-net-config until the pbr/pip snafu is resolved.22:17
bnemecSo the packaging work on that will likely not be completely finished until tomorrow.22:18
*** harlowja_ has quit IRC22:21
*** jamielennox|away is now known as jamielennox22:22
*** camunoz_gone is now known as camunoz22:23
*** teran has joined #openstack-infra22:25
*** Ryan_Lane has joined #openstack-infra22:27
*** teran_ has quit IRC22:29
*** carl_baldwin has quit IRC22:29
jamielennoxhi all, a call to review https://review.openstack.org/#/c/142954/ and https://review.openstack.org/#/c/142955/ - all keystoneclient and glanceclient reviews are currently failing gate.22:30
*** ryanpetrello has joined #openstack-infra22:30
*** Masahiro has joined #openstack-infra22:31
*** cburgess_ has joined #openstack-infra22:31
*** hdd has joined #openstack-infra22:34
*** prad has quit IRC22:34
*** sdake has quit IRC22:35
*** Masahiro has quit IRC22:36
*** ryanpetrello has quit IRC22:37
anteayajamielennox: this is your list of folks you need to bug: https://review.openstack.org/#/admin/groups/131,members22:37
jamielennoxanteaya: i have a feeling it's not all of them because it's a stable/icehouse review - i know i've bugged alex in the past and he only had core on the master branch22:38
*** EmilienM is now known as EmilienM|afk22:38
anteayaah22:39
jamielennoxi've no idea how to figure out that list22:39
anteayamight this be the stable maintenance teams then22:39
*** armax has quit IRC22:39
*** salv-orlando has quit IRC22:39
anteayahttps://review.openstack.org/#/admin/groups/535,members22:40
*** bswartz has joined #openstack-infra22:40
anteayahttps://review.openstack.org/#/admin/groups/538,members22:40
jamielennoxi'm sure it could be found in the projects repo22:40
anteayathe stable maintenance team recently became one team for each project22:40
morganfainbergyeah prob. stable maint22:41
sigmavirus24https://review.openstack.org/#/admin/projects/openstack/requirements,access shows the links to the stable-maint-core team: https://review.openstack.org/#/admin/groups/uuid-2267a5998d4224dd0acf1081eb2ee7b11573b7ea22:41
anteayamorganfainberg: you are stable maint for keystone22:41
morganfainberganteaya, yes22:41
jamielennoxsigmavirus24: ah! there we go22:41
sigmavirus24jamielennox: I've explored gerrit groups one too many times it seems =P22:42
sigmavirus24jamielennox: note release managers are also included in stable-maint-core so ttx and sdague can also +2 those patchsets22:42
anteayasdague: is on holidays22:42
*** reed has joined #openstack-infra22:42
anteayaand I'm going with ttx is asleep but probably will be around tomorrow22:43
adam_gjamielennox, im here and looking at those bugs, trying to grok whats going on22:43
anteayayay adam_g22:43
jamielennoxmost people are off somewhere - i've only got today so i figure it might be an after christmas problem22:43
sigmavirus24silly holidays22:43
sigmavirus24interfering with srsbsns22:43
*** salv-orlando has joined #openstack-infra22:44
jamielennoxadam_g:  so https://review.openstack.org/#/c/142974/1 is the dependant review22:45
jamielennoxironic-python-agent is in stable/icehouse global-requirements without version limits and the new version depends on oslo.concurrency22:45
grue_pmjamielennox: yeah a lot of big companies seem to have holidays last couple of weeks of december  and the first weekend in January22:46
adam_gjamielennox, i think it might be better to cap ironic-python-agent in stable/icehouse to < version that introduced the new dependency. or remove ironic-python-agent from stable/icehouse, im not sure why its there to begin with (the ironic agent didnt come around till juno)22:47
adam_gjroll, JayF ^22:47
jamielennoxadam_g: and here is an example of the keystoneclient failure22:47
jamielennoxhttp://logs.openstack.org/39/143339/1/check/gate-tempest-dsvm-neutron-src-python-keystoneclient-icehouse/22db298/logs/screen-s-proxy.txt.gz22:47
mtreinishjamielennox: yeah that was going to be my question too. What is causing ironic-python-agent to get pulled in for that job22:48
jamielennoxi didn't look into what was causing the keystoneclient 1.0.0 to be installed in the first place as 1.0.0 is compatible with the old one22:48
adam_gjamielennox, yeah, looking at those now. first time looking at those client library jobs so trying to wrap my head around it22:48
*** Guest87463 is now known as redrobot_away22:48
jamielennoxmtreinish: it's listed explicity as a icehouse project - it's not being pulled in: https://github.com/openstack/requirements/blob/stable/icehouse/projects.txt#L1722:49
jrolladam_g: was the agent driver even in ironic for icehouse release?22:49
adam_gjroll, not that i remember, thats why im wondering why its in stable/icehouse reqs :)22:49
jrollironic-python-agent was certainly under openstack/ at that time, though22:49
adam_ghm22:49
jrollI'm 99% certain it doesn't need to be there22:50
jrolltwo reasons:22:50
mtreinishjamielennox: ok, that's what I meant by pulled in. Ok so the issue is the latest version (or master) is pulling in oslo.concurrency which wasn't a thing back during icehouse22:50
jroll1) ironic-python-agent doesn't make sense to be in global requirements, it only makes sense in a ramdisk22:50
jamielennoxmtreinish: right22:51
jroll2) ironic didn't support using ironic-python-agent in icehouse22:51
jamielennoxmtreinish: just saying it's not a dependency22:51
jrollironic-python-agent is never a dependency for any project22:51
mtreinishjamielennox: ok I'd be hesitant to add it to the req list then. I think jroll is correct just drop the ironic agent from projects.txt seems like a better solution22:52
jrollalso, we haven't done releases for IPA, ever, so we'd have to push a tag or something to pin it22:52
mtreinishjroll: heh, well maybe it's time to push a tag for it :)22:53
jrollmtreinish: I think I agree, but does dropping it from projects.txt have any repercussions?22:53
mtreinishjroll: no, only if you tried to add a requirement to it which wasn't in the icehouse reqs list22:53
mtreinishbut since you're saying it's not used anywhere you can just say this isn't install compatible with the icehouse reqs and remove it22:54
jrollmtreinish: yep, makes sense to me22:54
jrolladam_g: ^ make sense to drop it/22:54
adam_g++22:54
jamielennoxok, i'll redo that review to drop ironic-python-agent instead of adding oslo.concurrency22:55
mtreinishjroll: fwiw: https://github.com/openstack/requirements/blob/master/README.rst#enforcement-in-projects22:55
jrollcool22:55
jrollthanks jamielennox22:55
*** FlorianSW has quit IRC22:55
adam_gmtreinish, do you know how the python-$fooclient stable jobs are meant to pass for clients that have been capped in stable?22:55
mtreinishadam_g: if they're capped this isn't an issue because the new reqs never try to be installed22:56
mtreinishadam_g: when they're uncapped we have the fun from a few weeks ago with all the oslo releases22:57
adam_gyeah22:57
adam_gim talking specifically about these client jobs22:57
adam_gpatches to master run the stable checks22:57
adam_gwhich AFAICS end up installing the client /w proposed patch as, say, python-glanceclient 0.15.0.post422:58
adam_gwhich may conflict with some stable project , ie 'but python-glanceclient>=0.9.0,!=0.14.0,<0.15 is required by ['ceilometer']'22:58
*** mfer has joined #openstack-infra22:58
jheskethkrtaylor, sweston: you guys highlighted me in a meeting re radar access22:59
dstufftfungi: 6.0.1 is out now22:59
jheskethdo we need to tweak some permissions there or need me to kick mikal or something?22:59
mtreinishadam_g: but isn't that separate, I think that error is the pep440 thing. Not a global reqs issue22:59
jamielennoxceilometer doesn't make that pinning, it's synced to global-requirements - it's just that ceilometer is the first thing that devstack installs22:59
fungidstufft: thanks! checking our mirrors, then rerunning a bunch of jobs23:00
mtreinishwell I mean it's an issue with the reqs listed in there not working with pep44023:00
swestonjhesketh: hi :-).  thanks for responding.  we were discussing core reviewer permissions on the radar repository.  can you help with that?23:01
dstufftfungi: virtualenv 12.0.1 will be out shortly too23:01
fungidstufft: oh, right. i need to wait on that actually23:01
*** mfer has quit IRC23:03
jamielennoxmtreinish: so keystoneclient and probably glanceclient are being retrieved from git - which seems like the problem for the icehouse integration tests, if pip was allowed to do the resolution it would probably work23:04
*** ryanpetrello has joined #openstack-infra23:05
jheskethsweston: I can probably bash mikal into adding people, I'm guessing since nobody is actually working on it and it's a stackforge project it probably doesn't matter that much23:05
adam_gjamielennox, for say,  https://review.openstack.org/#/c/120777/ , glanceclient is installed  from git because its checked out with the proposed patch. everything else comes from pip23:06
jheskethsweston: what are the gerrit username or emails for the people that need adding?23:06
*** sigmavirus24 is now known as sigmavirus24_awa23:07
adam_gwhat confuses me, is how a patch like that, proposed against master/0.15.0, could ever pass the stable check if that client is capped to <0.15.0 in one of the stable branch23:07
mtreinishjamielennox: oh, I see the issue now. install from git isn't respecting the cap in icehouse global-reqs23:07
adam_gis the intent to never cap clients and ensure compat via those jobs?23:07
mtreinishjamielennox: so devstack is installing it from git regardless of the cap being there23:07
mtreinishand the version exceeds the reqs cap causing it to explode.23:07
adam_gyeah23:07
jamielennoxmtreinish: adam_g is right - only the client being updated is being pulled from git, i just had a glanceclient and keystoneclient log open23:08
swestonjhesketh: thanks, I would appreciate any help you can offer.  gerrit username is steven-weston.23:08
adam_ghttp://git.openstack.org/cgit/openstack/requirements/tree/global-requirements.txt?h=stable/icehouse#n8123:08
swestonjhesketh: email is steve.weston@triniplex.com23:08
mtreinishjamielennox: so I would get around that by doing 1 of 2 things (probably both long term) 1. remove the icehouse job if there is a version cap for the client and the version is above the cap (it'll only get tricky if you push a backport to <0.12 to fix a bug on icehouse23:09
adam_gi guess anytime a client bumps its version to > oldest stable branch's cap, requirements cap needs to be relaxed?23:09
mtreinish2. fix devstack to check the cap in the install from git stuff that sdague added23:09
*** ryanpetrello has quit IRC23:09
*** ddieterly has quit IRC23:10
mtreinishadam_g: so the real fix is to have services in a venv, so that we can have a cap for installing clients for the services but not cap for testing the clients against running openstack23:10
*** otter768 has joined #openstack-infra23:11
fungimtreinish: real fix is to install different version of the client as a dep of the servers in a separate env from the client being exercised against the api, but we don't have that environment separation implemented yet23:11
*** ddieterly has joined #openstack-infra23:11
fungiright, that23:11
adam_gmtreinish, is that work still moving forward? i was catching only bits and pieces last week from the courthouse, and sounded like it was decided that venvs weren't really needed?23:11
jamielennoxmtreinish: the gate job should be asserting that the changes to the client don't break any backwards compatibility in icehouse - now that keystoneclient did a version bump it will never be below that cap again so the job is useless23:11
mtreinishbecause that's what we're struggling against here, is that we can't have too new a client running as part of icehouse, but we want to test that new clients work against icehouse23:11
mtreinishfungi: heh, isn't that what I said :)23:12
fungimtreinish: we were typing at the same time i think23:12
mtreinishfungi: dtroyer is working on installing the services inside of a venv, I saw a patch for it23:12
mtreinishI think23:12
fungijamielennox: the trick is being able to run two separate versions of the client in the job at the same time23:12
fungiin parallel23:12
fungithe capped version and the patch being tested23:12
adam_gmtreinish, is the goal of those jobs to use the newer clients /w tempest right?23:12
mtreinishjamielennox: yes, that was #1 on my short term work arounds23:12
openstackgerritJoshua Hesketh proposed openstack-infra/system-config: Remove python magic  https://review.openstack.org/14233023:13
jamielennoxfungi: why wouldn't we want to test the new client version on all services?23:13
mtreinishadam_g: yeah, although I don't want the official clients in tempest anymore (they're only used for cli testing which doesn't belong in tempest anymore)23:13
jamielennoxie ignore the cap for this patch and run icehouse with the master keystoneclient23:13
mtreinishadam_g: the clients need there own functional testing really23:13
mtreinishbut that's separate :)23:13
fungijamielennox: because the client might be a dependency of a stable release service, and the newer version of the client might have dependencies of its own which are incompatible with the dependencies of the server in question23:14
*** teran has quit IRC23:14
dtroyermtreinish, fungi: I am working on using a global venv in DevStack, it's a whack-a-mole problem making everything work…23:15
dtroyerthis is one reason why we really need API libs with as absofrickinlutely the fewest number of dependencies as possible...23:16
jamielennoxhm, however if we don't test the new client everywhere we're reducing the point of the gate job23:16
*** ddieterly has quit IRC23:16
*** teran has joined #openstack-infra23:16
*** amitgandhinz has quit IRC23:16
fungias a dependent lib, true, but as an api client not necessarily as long as its interactions are exercised fairly thoroughly23:17
mtreinishfungi: heh, which most really aren't, but that's a separate issue :)23:18
fungiyep23:18
jamielennoxyou mean as a CLI app? devstack uses openstackclient so they're not being tested23:18
jamielennox(at least for keystone)23:18
fungijamielennox: well, at the moment openstackclient still uses the client libs, unless someone's already solved that23:18
fungithough tempest has its own lightweight api client as well23:19
fungiwhich is i think what mtreinish was alluding to23:19
dtroyerfungi: not yet… ;)23:19
mtreinishjamielennox: that's the separate issue (tempest does do python-*client cli testing fwiw, but that needs to be moved into the client repos)23:19
*** hichtakk has quit IRC23:19
*** hichtakk has joined #openstack-infra23:19
mtreinishjamielennox: the client libs need to have their own functional testing, which is a current gap23:19
*** Ryan_Lane has quit IRC23:20
jogois there a bug filed for the pip 6.0 related issues (for e-r etc.23:20
mtreinishjogo: I don't think so, at least I never bothered to23:21
fungijogo: good luck trying to sort through and identify discrete issues23:22
jamielennoxmtreinish: that will solve it longer term23:22
jogofungi: fingerprints can be many ORs23:22
mtreinishjamielennox: heh, hence my short term suggestion of just dropping the job if the current version exceeds the stable cap23:23
*** andreaf has quit IRC23:23
openstackgerritSpencer Krum proposed openstack-infra/puppet-storyboard: bump rabbitmq to 5.0  https://review.openstack.org/14356023:23
fungijogo: fair enough. if you have several you want to associate with that i'm happy to let you know if they match things i've seen/dealt with so far23:23
jogofungi: cool, will look into after my run23:23
jamielennoxmtreinish: that's always going to be the case23:23
anteayajogo: OR operatring room, over reaching, open ridicule23:23
*** andreaf has joined #openstack-infra23:23
fungianteaya: or a capitalized conjunction used as a logical operator23:24
anteayaah ha23:24
jogoso operating room OR over reaching OR open ridicule ;)23:24
fungibwahaha23:24
anteayaperfect23:25
adam_gif functional testing moved into client libs as well as servers, dtroyer's global devstack venv might work fine there. devstack sets everything up in a venv, tests run outside /w newer set of client lib's master requirements23:25
*** ociuhandu has quit IRC23:25
dstufftfungi: 12.0.1 cut23:25
jamielennoxadam_g: right23:26
fungidstufft: thanks again! i'll let you know (or properly open an issue) if i spot anything new23:26
mtreinishjamielennox: well until we can do the separate venvs for services then the issue goes away because we can respect the cap and test the new version23:26
nibalizerkrotscheck: ^^ fix for all the rabbitmq problems23:26
*** reazem has joined #openstack-infra23:27
jamielennoxmtreinish: so maybe the answer is the icehouse gate job should just be removed because anything in icehouse should be capped to not allow the current client lib to be used anyway23:28
jamielennoxadam_g: ^23:28
adam_gjamielennox, yeah, looking at the zuul layout now23:28
mtreinishjamielennox: yes, that was what I was suggesting. But only do it for clients that exceed the cap. If the current working version will still get pulled in the job needs to stick around23:30
jheskethsweston: cool, I'll ping mikal. He is on vacation though so it may take a day or so23:30
fungiokay, virtualenv 12.0.1 is in our mirrors now. rechecking the devstack juno fix23:30
jamielennoxas a small sample there is a gate-tempest-dsvm-neutron-src-python-X-icehouse job in keystoneclient, glanceclient, novaclient, cinderclient23:30
mtreinishadam_g: I think it might just be a template right now, which adds both icehouse and juno23:30
adam_gmtreinish, yup, found it23:30
swestonjhesketh: no worries, thank you so much :-)23:30
mtreinishjamielennox: it'll be on all the client libs, and all oslo libs.23:31
mtreinishjamielennox: basically it should be on anything that has the option to isntall a lib from git in devstack23:31
openstackgerritAdam Gandelman proposed openstack-infra/project-config: Remove icehouse compat checks from glance/keystoneclient  https://review.openstack.org/14356623:33
adam_gmtreinish, jamielennox ^23:33
adam_goh oh, we want icehouse removed from all the clients?23:35
*** dims has quit IRC23:35
mtreinishadam_g: I don't think so, only if they exceed an icehouse cap23:35
jamielennoxi think as it becomes required23:35
adam_gack23:35
adam_ghttp://paste.ubuntu.com/9599918/23:35
jamielennoxso juno doesn't pin maximum versions, post kilo we'll need to change these back to all compat23:36
*** ddieterly has joined #openstack-infra23:36
adam_gyeah, or get the venv stuff in place first23:37
dstufftfungi: is there a link for the devstack juno fix23:37
jamielennoxok, abandoning the version bump reviews in favour of the zuul change23:38
*** sdake has joined #openstack-infra23:39
fungidstufft: https://review.openstack.org/14350423:39
* dstufft finds it amusing that people apparently care that pip is 6.0 and not 1.623:39
fungidstufft: once that merges, then https://review.openstack.org/143501 should be passing23:39
fungidstufft: i tried to come up with a > joke for that, but i'm running out of caring23:40
*** Katherine has joined #openstack-infra23:41
dstufftfungi: lol23:42
*** armax has joined #openstack-infra23:42
*** dmsimard is now known as dmsimard_away23:44
*** dmsimard_away is now known as dmsimard23:44
*** stevemar has quit IRC23:45
*** zz_zz_zz_zz_saba is now known as zz_zz_zz_zz_zz_s23:46
*** reed has quit IRC23:50
*** reed has joined #openstack-infra23:52
asselinThis nodepool image-update get-pip error just started happening. Anyone seen it before? http://paste.openstack.org/show/154013/23:52
*** e0ne has quit IRC23:52
asselinAttributeError: 'module' object has no attribute 'command_dict'23:53
Katherinehi all.  katherine here again.  i'm hoping someone can answer a really basic zuul question for me23:54
jeblairdstufft: ^ see message from asselin; i just grabbed git-pip and ran it and got the same thing23:55
asselinfungi, ^^ get-pip.py                                         22-Dec-2014 23:45             158107523:55
Katherinezuul checks for dependencies in gerrit reviews and tests those dependent ones together.  it also cancels all gate jobs currently running if there is an outstanding review that will conflict with the review-under-test23:55
Katherinehow does zuul determine if reviews have dependencies on one another?23:55
jeblairKatherine: based on git commit dependencies23:56
jeblairKatherine: so if a commit in change A depends on a commit in change B, zuul will test A on top of B.  Gerrit uses the same info to show dependencies in the ui23:57
jeblairKatherine: so if a change shows up in gerrit as depending on another change in gerrit, it will in zuul as well23:57
Katherinejeblair: so if reviews have common file changes in common?23:57
dstufftasselin: jeblair try now23:57
fungiasselin: jeblair: dstufft: the bootstrap.pypa.io site claims get-pip.py was updated ~13 minutes ago23:58
jeblairKatherine: no -- it's based on a commit in git being the parent of another commit in git23:58
fungioh, weird, now it says 8 hours ago23:58
dstufftfungi: if you're looking at the / listing that's not up to date with what I just pushed23:58
* asselin trying23:58
Katherinejeblair: so say i push a change out for review that modifies a method and another developer pushes a review that removes that method, the two are dependent but not explicitly so23:58
dstufftfungi: bootstrap.pypa.io is served by Fastly23:59
fungidstufft: aha, so it could be random23:59
dstufftand we don't bother to purge /23:59
*** Ryan_Lane has joined #openstack-infra23:59
asselindstufft, negative23:59
dstufftasselin: same error?23:59
asselinyes23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!