Friday, 2015-06-19

*** sdake_ has joined #openstack-infra00:00
*** ZZelle_ has joined #openstack-infra00:01
*** ZZelle_ has quit IRC00:01
jeblairmordred, fungi: the gozer folks encountered an error that i think may be responsible for the 'nodepool demand outage' that we saw this morning during uploads00:01
jeblairi believe it is fixed by https://review.openstack.org/19336800:02
jeblairSpamapS: ^ you may be interested too00:02
mordredjeblair: I am in favor of bugfixe00:02
mordredjeblair: I am in favor of bugfixes00:02
jeblairthis exception is in our geard log file: http://paste.openstack.org/show/303024/00:02
*** Guest44457 has quit IRC00:03
openstackgerritSteven Dake proposed openstack-infra/project-config: Revert "Make kolla functional job voting"  https://review.openstack.org/19336900:03
jeblairnote, i have not run zuul tests with that yet (we should set up a x-gate job with that)00:03
jeblairwhich is something that should be done before we try that in prod00:03
jeblairi have to run now though00:03
*** rcarrillocruz has joined #openstack-infra00:03
*** sdake has quit IRC00:03
sdake_could a couple infra cats approve that revert - our gating software apparenty doesn't work well with both a check and gate at the same time00:04
*** sarob has quit IRC00:04
SpamapSjeblair: quite interesting00:04
*** puranamr has quit IRC00:04
jeblairSpamapS: yeah, it's kind of in the "how did that ever work department" and then, as we found out this morning, perhaps it doesn't work.  :)00:05
greghaynesoh - so are we filling up the TCP buffers while were sending data to rax?00:05
greghayneswhich uncovered this buyg?00:05
jeblairgreghaynes: i think so00:05
greghaynesawesome!00:05
SpamapSjeblair: we should run some libhostile testing on geard btw. ;)00:05
SpamapSjeblair: http://libhostile.readthedocs.org/en/latest/00:05
sdake_by "our gating softwre" I mean kollas, not infras00:06
*** puranamr has joined #openstack-infra00:06
*** sdake_ is now known as sdake00:06
*** puranamr has quit IRC00:07
*** rlandy has quit IRC00:09
greghaynesmordred: hrm, so looks like glean is just not writing out any net config at all. \O/00:09
greghaynesmordred: I also forgot to give my console user sude perms, so waiting on a respin00:10
*** rcarrillocruz has quit IRC00:11
mordredgreghaynes: oh good!00:11
mordredgreghaynes: (you remembered to boot your vm with config-drive attached, right?)00:12
greghaynesyep00:12
greghaynessaw the config drive00:12
*** armax has quit IRC00:16
*** rcarrillocruz has joined #openstack-infra00:17
*** olaph has quit IRC00:17
openstackgerritMerged openstack-infra/system-config: Enable logging on openstack-app-catalog IRC channel  https://review.openstack.org/19034300:18
*** miqui has quit IRC00:19
openstackgerritlifeless proposed openstack-dev/pbr: Strip markers from test requirements.  https://review.openstack.org/19337200:19
*** wenlock has quit IRC00:19
*** erlon has quit IRC00:20
*** sarob has joined #openstack-infra00:20
openstackgerritlifeless proposed openstack/requirements: Remove hacking and hacking-owned dependencies.  https://review.openstack.org/19292900:20
openstackgerritlifeless proposed openstack/requirements: Mark as Python2.7 only the dependencies that are.  https://review.openstack.org/18925000:20
openstackgerritlifeless proposed openstack/requirements: Add upper-constraints.txt.  https://review.openstack.org/18925100:20
*** vnod has quit IRC00:21
*** amotoki_ has joined #openstack-infra00:23
*** sabeen3 has quit IRC00:24
*** yamamoto has joined #openstack-infra00:24
*** hdd has quit IRC00:27
*** gyee has quit IRC00:27
*** amotoki_ has quit IRC00:28
*** yamamoto has quit IRC00:29
sdakethe kolla review queue is completely blocked  atm, could couple infra cores approve this revert : https://review.openstack.org/19336900:30
*** mtanino has quit IRC00:31
lifelessmordred: how did you go reviewing my stack ?00:33
lifelessmordred: I see less +2's than I'd hoped :/00:33
*** r-daneel has quit IRC00:34
*** weshay has joined #openstack-infra00:35
*** dims has joined #openstack-infra00:39
*** sdake_ has joined #openstack-infra00:41
*** abitha has quit IRC00:42
*** sdake has quit IRC00:44
*** dims has quit IRC00:47
*** dims has joined #openstack-infra00:47
openstackgerritEmilien Macchi proposed openstack-infra/project-config: zuul: actually enable puppet4 voting  https://review.openstack.org/19337900:48
openstackgerritJerry Zhao proposed openstack-infra/nodepool: add option to use ipv6 for image update and node launching  https://review.openstack.org/15617800:51
*** koolhead17 has joined #openstack-infra00:52
*** koolhead17 has quit IRC00:52
*** koolhead17 has joined #openstack-infra00:52
*** weshay has quit IRC00:53
*** openstack has joined #openstack-infra00:56
*** markvoelker has joined #openstack-infra00:56
*** alexsyip has quit IRC00:59
*** markvoelker has quit IRC01:02
*** hodos has quit IRC01:02
openstackgerritMatt Riedemann proposed openstack/requirements: Add glance_store to projects.txt  https://review.openstack.org/18268101:04
*** mriedem_afk is now known as mriedem01:09
*** bknudson has joined #openstack-infra01:10
*** enikanorov2 has quit IRC01:10
*** yamamoto has joined #openstack-infra01:14
*** vaidy_ has joined #openstack-infra01:15
*** yamamoto has quit IRC01:18
*** ivar-lazzaro has quit IRC01:19
*** otter768 has joined #openstack-infra01:22
*** enikanorov2 has joined #openstack-infra01:23
openstackgerritKyle Mestery proposed openstack-infra/project-config: Take configuraiton control to test hooks for networking-odl  https://review.openstack.org/19243301:23
*** otter768 has quit IRC01:26
*** EricGonczer_ has joined #openstack-infra01:27
*** zz_dimtruck is now known as dimtruck01:28
*** woodster_ has quit IRC01:31
*** Longgeek has joined #openstack-infra01:31
*** tlian has joined #openstack-infra01:33
*** ddieterly has joined #openstack-infra01:33
*** EricGonczer_ has quit IRC01:38
lifelessfungi: 189250 got dims +101:39
lifelessfungi: ( I tweaked a couple entries subsequently )01:39
*** clu_ has quit IRC01:39
*** samueldmq has joined #openstack-infra01:39
fungilifeless: cool. i take it you're still wrestling with devstack there?01:42
lifelessfungi: I added a depends-on to a pbr change01:42
fungiahh, keen01:42
openstackgerritJohn Trowbridge proposed openstack-infra/project-config: add stackforge/ahc-tools  https://review.openstack.org/19339201:42
lifelessbut even if it passes via depends-on, I don't think we should merge it until pbr is released with that fix01:43
lifelessright now I'm poking at the new deps proton needs01:43
*** dkranz has joined #openstack-infra01:43
lifelessto make generate-constraints work locally01:43
lifeless    swig -python -threads -o cproton_wrap.c cproton.i01:43
lifeless    cproton.i:310: Error: Unable to find 'proton/cproton.i'01:43
lifeless    error: command 'swig' failed with exit status 101:43
lifelessfungi: 193372 is the pbr change01:43
lifelessfungi: which appeard to run its devstack run with the transiently broken pbr thing which has now landed01:44
lifelessfungi: so should be fine (now)01:44
fungiyeah, i'm looking at the test logs from the pbr change01:45
lifelessI've told it to recheck01:45
fungi/opt/stack/new/devstack/inc/python: line 230: ./.venv/bin/python: No such file or directory01:46
lifelessyes01:46
lifelessthat was the transient breakage01:46
lifelesssee devstack master git log01:47
lifelessdo you happen to know what packages we need to add to install python-proton-qpid ?01:47
lifelessI've adde uuid-dev swig01:47
lifelessbut its barfing01:47
lifelessas above01:47
lifelessflaper87: ^01:47
fungiaha, yep01:48
fungilifeless: i think they were recently added to the bare-.* workers. checking git history now01:49
lifelessfungi: they were added to tools/integration.sh01:50
lifelesshmm, wonder if its python301:50
fungilifeless: on debuntu it needs uuid-dev and swig01:50
fungion frhelos it's libuuid-devel instead of uuid-dev01:50
*** Longgeek has quit IRC01:50
lifelessyeah01:51
lifelessit fails on python 3.401:51
lifelesshahahahahaaha01:51
fungiso nice01:51
* lifeless blacklists it01:51
fungilifeless: at least proton is old and being retired, right?01:51
*** laughterwym has joined #openstack-infra01:52
lifelesswell01:54
lifelessthis is the new shiny one01:54
lifelessapparently01:54
*** tjones1 has joined #openstack-infra01:54
lifelessflaper has been doing some work to make it less awful01:54
lifelessbut yes, the pyngus update needs to be version capped01:55
fungioh, found https://issues.apache.org/jira/browse/PROTON-49001:55
lifelessso I'm thinking we probably want to run generate-constraints during CI of changes to openstack/requirements01:56
lifelessbecause new things that come in might be wrongly versioned01:56
lifelesserm01:56
lifelessmarkered01:56
lifelessand we won't catch that until the periodic job today01:56
* fungi was also kidding about proton being retired. knows it's recent addition01:56
*** vaidy_ has quit IRC01:56
*** sarob has quit IRC01:57
*** markvoelker has joined #openstack-infra01:57
openstackgerritlifeless proposed openstack/requirements: Remove hacking and hacking-owned dependencies.  https://review.openstack.org/19292901:58
openstackgerritlifeless proposed openstack/requirements: Mark as Python2.7 only the dependencies that are.  https://review.openstack.org/18925001:58
openstackgerritlifeless proposed openstack/requirements: Add upper-constraints.txt.  https://review.openstack.org/18925101:58
*** enikanorov2 has quit IRC01:59
lifelessmeh, restart all your CIs. :/01:59
openstackgerritlifeless proposed openstack-dev/pbr: Strip markers from test requirements.  https://review.openstack.org/19337201:59
*** nijaba has joined #openstack-infra02:00
*** tsufiev has quit IRC02:01
*** ddieterly has quit IRC02:02
*** yamamoto has joined #openstack-infra02:02
*** markvoelker has quit IRC02:02
*** tsufiev has joined #openstack-infra02:06
openstackgerritMerged openstack-infra/tripleo-ci: Don't sanitize node names for log directories  https://review.openstack.org/19310402:11
*** rlandy has joined #openstack-infra02:15
bhunter71hi all, a question.  When post/release/tag zuul pipelines fail jobs, how does one know?  Does it email someone?   (I only know by watching my zuul UI)02:24
*** amotoki_ has joined #openstack-infra02:24
*** amotoki_ has quit IRC02:30
*** claudiub has quit IRC02:31
*** ddieterly has joined #openstack-infra02:33
*** sdake_ has quit IRC02:34
*** prad has quit IRC02:35
*** marzif has quit IRC02:36
*** ddieterly has quit IRC02:37
*** AzherKhan has joined #openstack-infra02:38
*** sabeen has joined #openstack-infra02:39
*** tjones1 has quit IRC02:39
*** cburgess has quit IRC02:39
*** hdd has joined #openstack-infra02:41
*** cburgess has joined #openstack-infra02:44
*** rlandy has quit IRC02:45
*** rlandy has joined #openstack-infra02:45
*** jamielennox is now known as jamielennox|away02:46
*** Longgeek has joined #openstack-infra02:47
*** jamielennox|away is now known as jamielennox02:53
lifelessfungi: 193372 is green02:57
lifelessmordred: ^02:57
lifelessthats a critical-path fix for pbr02:57
*** stevemar has joined #openstack-infra03:01
*** wuhg has joined #openstack-infra03:03
*** cburgess has quit IRC03:04
*** cburgess has joined #openstack-infra03:05
*** dims has quit IRC03:06
*** cburgess has quit IRC03:07
*** cburgess has joined #openstack-infra03:10
*** rlandy has quit IRC03:12
*** rlandy has joined #openstack-infra03:13
*** bhunter71 has quit IRC03:13
*** koolhead17 has quit IRC03:14
*** armax has joined #openstack-infra03:16
*** tlian has quit IRC03:17
*** Longgeek has quit IRC03:19
*** stevemar has quit IRC03:22
*** EricGonczer_ has joined #openstack-infra03:22
*** otter768 has joined #openstack-infra03:23
*** rm_work has quit IRC03:25
*** EricGonczer_ has quit IRC03:25
*** amotoki_ has joined #openstack-infra03:25
*** stevemar has joined #openstack-infra03:26
*** otter768 has quit IRC03:28
*** coolsvap|away is now known as coolsvap03:30
*** amotoki_ has quit IRC03:30
*** rm_work|away has joined #openstack-infra03:31
*** rm_work|away is now known as rm_work03:31
*** rm_work has joined #openstack-infra03:31
*** crc32 has quit IRC03:31
*** sdake has joined #openstack-infra03:33
*** dimtruck is now known as zz_dimtruck03:37
*** ashleighfarnham has joined #openstack-infra03:42
*** markvoelker has joined #openstack-infra03:46
*** markvoelker has quit IRC03:51
*** mriedem has quit IRC03:55
*** ildikov_ has quit IRC03:55
*** koolhead17 has joined #openstack-infra04:00
*** koolhead17 has quit IRC04:00
*** koolhead17 has joined #openstack-infra04:00
*** Longgeek has joined #openstack-infra04:00
*** rlandy has quit IRC04:02
sdakehttps://www.centos.org/forums/viewtopic.php?t=142004:04
sdakeis it possible some chnge in infra affects the setting of the system clocks in a negative way?04:04
*** zz_jgrimm has quit IRC04:05
*** zz_ja has quit IRC04:05
*** maurosr has quit IRC04:05
*** Longgeek has quit IRC04:06
*** zz_jgrimm has joined #openstack-infra04:06
*** maurosr has joined #openstack-infra04:06
*** vnod has joined #openstack-infra04:07
*** zz_ja has joined #openstack-infra04:07
*** sdake_ has joined #openstack-infra04:07
*** Longgeek has joined #openstack-infra04:08
*** abitha has joined #openstack-infra04:10
*** sdake has quit IRC04:10
*** abitha has quit IRC04:10
*** abitha has joined #openstack-infra04:11
*** Longgeek_ has joined #openstack-infra04:12
*** fedexo has joined #openstack-infra04:12
*** Longgeek has quit IRC04:15
*** harlowja_at_home has joined #openstack-infra04:21
*** ashleighfarnham has quit IRC04:26
*** amotoki_ has joined #openstack-infra04:26
*** vivek_v_c has joined #openstack-infra04:27
*** camunoz is now known as camunoz_mtg04:30
*** rossella_s has quit IRC04:30
*** rossella_s has joined #openstack-infra04:31
*** amotoki_ has quit IRC04:31
*** vivek_v_c is now known as vivcheri04:31
*** achanda has joined #openstack-infra04:33
*** rm_you has joined #openstack-infra04:37
*** koolhead17 has quit IRC04:40
*** jtomasek has joined #openstack-infra04:42
*** markvoelker has joined #openstack-infra04:47
*** markvoelker has quit IRC04:51
*** achanda has quit IRC04:59
ianwsdake: devstack-gate should get the time right -> http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n11805:00
sdake_ianw thanks05:03
sdake_but we dont actually run devstack ;)05:04
*** ddieterly has joined #openstack-infra05:04
ianwsdake_: ok, what's the problem then?05:05
sdake_ianw we download a bunch of stuff via epel (https)05:07
sdake_we do it about 40-50 times separately incontainers05:07
sdake_sometimes they fail05:07
sdake_the failure rate has gone from 1-5% previously to 100% today05:07
*** jog0 has joined #openstack-infra05:07
sdake_ianw internet googling points to time synchronization issues resulting in the 500 errors coming out of the mirror list via the https protocol05:08
sdake_ianw or firewalls on port 44305:08
*** ddieterly has quit IRC05:09
sdake_since we dont run devstack we have our own gate setup05:09
sdake_ianw https://github.com/stackforge/kolla/blob/master/tests/setup_gate.sh05:09
sdake_I guess one quetion I'd have is why is this spiraled out of control today05:11
sdake_we made our gate from nonvoting to voting05:11
sdake_but I dont think that is the immediate issue05:12
*** boris-42 has quit IRC05:12
ianwsdake_: well, could be, i've certainly seen yum go a bit crazy when restoring a snapshot image that has a very wrong time05:15
*** sabeen has quit IRC05:17
*** abitha has quit IRC05:18
*** garyk has joined #openstack-infra05:19
*** rm_you has quit IRC05:19
*** otter768 has joined #openstack-infra05:24
*** camunoz_mtg is now known as camunoz05:24
*** amotoki_ has joined #openstack-infra05:27
*** otter768 has quit IRC05:28
*** harlowja_at_home has quit IRC05:29
*** yamada-h has joined #openstack-infra05:31
*** amotoki_ has quit IRC05:31
*** jog0 has quit IRC05:32
*** AzherKhan has quit IRC05:33
*** sdake_ has quit IRC05:33
*** BharatK has joined #openstack-infra05:48
*** puranamr has joined #openstack-infra05:48
*** sarob has joined #openstack-infra05:56
*** vnod has quit IRC05:56
*** maishsk_afk has quit IRC06:01
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/19297906:01
*** maishsk_afk has joined #openstack-infra06:03
*** laughterwym has quit IRC06:05
*** ddieterly has joined #openstack-infra06:05
AJaegerStevenK: good morning!06:06
StevenKAJaeger: Hai!06:06
AJaegeryour scripts are life and seems they fail - did you see that already?06:07
*** Kennan2 has joined #openstack-infra06:07
*** Kennan has quit IRC06:07
AJaegerGo to https://jenkins.openstack.org/ and check the "upstream translation update" output, e.g. https://jenkins.openstack.org/job/oslo.db-upstream-translation-update/18/consoleFull06:08
*** maishsk_afk has quit IRC06:08
*** hdd has quit IRC06:09
StevenKAJaeger: Yup, I knew they would fail, since other parts haven't landed, but they do not impact the running of the script06:09
AJaegerStevenK: Indeed,...06:10
*** stevemar has quit IRC06:10
StevenKSo it's all good, the scripts will just be a little noisier06:10
*** ddieterly has quit IRC06:10
AJaegerOk, then let's get the other parts in06:10
AJaegerYeah, nobody besides the two of us looks at that noise ;)06:11
AJaegerRegarding horizon, did you see https://review.openstack.org/#/c/188665 - I doubt it will help with your scripting work but wanted you to be aware of it06:11
StevenKAJaeger: Requires hiera setup, pleia2 is on it06:11
*** maishsk_afk has joined #openstack-infra06:12
StevenKOh, and zanata server setup06:12
StevenKAJaeger: Yes, I don't believe those changes will impact my script work06:12
*** ParsectiX has joined #openstack-infra06:15
*** maishsk_afk has quit IRC06:16
*** flepied1 has joined #openstack-infra06:20
AJaegerStevenK: That's my impression as well. Initially I thought they might. But them going to babel might make life easier in some areas if you need to make changes there.06:21
*** armax has quit IRC06:22
*** sarob has quit IRC06:22
*** puranamr has quit IRC06:22
StevenKAJaeger: My current issue is that Zanata won't deal with django{,js}.po -- I have to symlink it to a .pot file :-(06:23
*** e0ne has joined #openstack-infra06:25
*** mrunge has joined #openstack-infra06:25
AJaegerStevenK: can we rename that in the repos?06:26
AJaegerStevenK: moving forward: We need to handle translation of stable branches as well. But that's a complete new feature and one the translators would really love AFAIU.06:27
AJaegerright now we do not handle branches at all with our scripts06:27
StevenKAJaeger: Zanata can in fact do that, we can handle multiple versions06:27
StevenKAt the moment, we have one version, 'master', but we could certainly have a job that creates a 'liberty' version, and then pushes and pulls from that06:28
StevenKAJaeger: So, we could, except for the fact that the proposal scripts have the current path hardcoded06:29
*** e0ne has quit IRC06:31
*** jtomasek has quit IRC06:32
*** k4n0 has joined #openstack-infra06:34
*** markvoelker has joined #openstack-infra06:36
openstackgerritMerged openstack-infra/irc-meetings: Revert "Fix incorrect docs meeting time"  https://review.openstack.org/19322406:37
openstackgerritMerged openstack-infra/irc-meetings: Change name of calendar dir to 'calendars'  https://review.openstack.org/19312306:38
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/19297906:39
*** markvoelker has quit IRC06:40
openstackgerritWayne Warren proposed openstack-infra/infra-specs: Jenkins Job Builder 2.0.0 API Changes/Rewrite  https://review.openstack.org/18254206:41
*** koolhead17 has joined #openstack-infra06:43
*** juggler has quit IRC06:43
AJaegerStevenK: yeah, something for the future. Just wanted to remind you of that since you're working on the scripts...06:45
openstackgerritTony Breeds proposed openstack-infra/irc-meetings: Add tricircle meeting  https://review.openstack.org/19179306:45
*** maishsk_afk has joined #openstack-infra06:47
*** koolhead17 has quit IRC06:48
StevenKAJaeger: Yeah, that may require a smaller spec, or at least concensus on the mailing list, but let's get master working first. :-)06:49
AJaegerStevenK: agreed on both points06:49
*** koolhead17 has joined #openstack-infra06:49
*** mpaolino has joined #openstack-infra06:54
*** fedexo has quit IRC06:55
AJaegerlifeless: are you around and can help with library releases?06:58
*** AzherKhan has joined #openstack-infra07:00
StevenKAJaeger: Re Horizon and .pot files, I think I'll draft a mailing list post on my Monday and see what the Horizon folk say before writing up a patch07:03
StevenKBecause there are number of ways we could handle it07:04
*** jlanoux has joined #openstack-infra07:04
AJaegerStevenK: ok...07:05
*** camunoz has quit IRC07:05
*** oomichi has joined #openstack-infra07:05
*** oomichi_ has joined #openstack-infra07:05
*** ddieterly has joined #openstack-infra07:06
*** chlong has quit IRC07:06
*** ddieterly has quit IRC07:10
*** markus_z has joined #openstack-infra07:16
*** Hal has joined #openstack-infra07:17
*** ihrachyshka has joined #openstack-infra07:17
*** Hal is now known as Guest1091707:17
*** jlanoux has quit IRC07:18
*** jlanoux has joined #openstack-infra07:19
*** dizquierdo has joined #openstack-infra07:23
*** oomichi has quit IRC07:24
*** oomichi_ has quit IRC07:24
*** otter768 has joined #openstack-infra07:24
*** ashleighfarnham has joined #openstack-infra07:26
*** devvesa has joined #openstack-infra07:26
lifelessAJaeger: I am07:27
lifelessAJaeger: but part of the decision in vancouver was to release very early in the week07:27
lifelessAJaeger: friday is not early :)07:27
AJaegerlifeless: I wasn't aware that "internal tools" were part of the library release.07:28
lifelessAJaeger: this is the docs thing?07:28
AJaegerlifeless: I send you and dhellmann an email asking for a release in the mean time.07:28
AJaegerlifeless: yes. I'm fine having it handled by you, it's just changing my workflow ;)07:28
*** Ala has joined #openstack-infra07:29
*** otter768 has quit IRC07:29
AJaegerIf you batch release early next week, it's fine for me as well.07:29
*** deepakcs has joined #openstack-infra07:30
*** ashleighfarnham has quit IRC07:30
lifelesslet me reply on the list07:32
AJaegerlifeless: sure, thanks!07:33
*** _nadya_ has joined #openstack-infra07:34
*** otherwiseguy has quit IRC07:36
*** notnownikki has joined #openstack-infra07:36
AJaegerlifeless: I see ttx answered as well my mail (thanks) - so, let's continue via email07:37
*** dtantsur|afk is now known as dtantsur07:45
*** maishsk_afk has quit IRC07:49
*** maishsk_afk has joined #openstack-infra07:56
*** claudiub has joined #openstack-infra07:58
*** jamielennox is now known as jamielennox|away08:00
*** hashar has joined #openstack-infra08:01
*** otherwiseguy has joined #openstack-infra08:02
*** ZZelle has quit IRC08:03
*** ZZelle has joined #openstack-infra08:03
*** jcoufal has joined #openstack-infra08:06
*** ddieterly has joined #openstack-infra08:06
*** arxcruz has joined #openstack-infra08:11
*** ddieterly has quit IRC08:11
*** bexelbie_ has joined #openstack-infra08:14
*** hashar has quit IRC08:15
*** ildikov_ has joined #openstack-infra08:15
*** cdent has joined #openstack-infra08:16
*** yamada-h has quit IRC08:16
*** boris-42 has joined #openstack-infra08:17
*** e0ne has joined #openstack-infra08:18
*** derekh has joined #openstack-infra08:18
*** hashar has joined #openstack-infra08:23
*** nihilifer has joined #openstack-infra08:25
*** markvoelker has joined #openstack-infra08:25
*** afazekas has joined #openstack-infra08:25
*** e0ne has quit IRC08:25
*** amotoki_ has joined #openstack-infra08:29
*** markvoelker has quit IRC08:29
*** yamada-h has joined #openstack-infra08:29
*** alexpilotti has joined #openstack-infra08:33
*** hashar has quit IRC08:34
*** amotoki_ has quit IRC08:34
*** hashar has joined #openstack-infra08:34
*** hichihara has quit IRC08:35
*** yamada-h has quit IRC08:39
*** vivcheri has quit IRC08:41
*** devvesa has quit IRC08:46
*** hodos has joined #openstack-infra08:49
*** Longgeek_ has quit IRC08:51
*** Longgeek has joined #openstack-infra08:51
nihiliferhello08:57
nihilifercan someone look at this patch? https://review.openstack.org/#/c/189853/08:57
nihiliferit already had +2 yesterday, but needed rebase08:57
nihilifersdauge: could you take a look?08:59
*** aarefiev has quit IRC09:00
*** devvesa has joined #openstack-infra09:01
*** masayukig_ has quit IRC09:01
*** svasheka has quit IRC09:01
*** sslypushenko has quit IRC09:02
*** romcheg has joined #openstack-infra09:02
*** masayukig_ has joined #openstack-infra09:02
*** wznoinsk has quit IRC09:03
*** dtantsur is now known as dtantsur|brb09:03
*** wznoinsk has joined #openstack-infra09:04
*** vponomaryov has quit IRC09:05
*** aarefiev has joined #openstack-infra09:06
*** ddieterly has joined #openstack-infra09:07
openstackgerritThomas Bechtold proposed openstack-dev/pbr: Remove sphinx_config.init_values() manual call  https://review.openstack.org/19346209:10
*** vponomaryov has joined #openstack-infra09:11
*** ddieterly has quit IRC09:12
*** devvesa has quit IRC09:13
*** juggler has joined #openstack-infra09:14
*** romcheg has quit IRC09:21
*** Hal has joined #openstack-infra09:21
*** Guest10917 has quit IRC09:21
*** Hal is now known as Guest1659809:21
*** otter768 has joined #openstack-infra09:25
flip214Hi. Is there some way I can get a list of recent CI runs (for drbd-devstack) with the SUCCESS/FAILURE information and a link to the results? Perhaps even statistics can be queried?09:26
*** dizquierdo has quit IRC09:27
*** e0ne has joined #openstack-infra09:28
*** devvesa has joined #openstack-infra09:29
*** coolsvap is now known as coolsvap|away09:30
*** _nadya_ has quit IRC09:30
*** amotoki_ has joined #openstack-infra09:30
*** Guest16598 has quit IRC09:30
*** otter768 has quit IRC09:30
Kiallflip214: http://graphite.openstack.org/ has stats etc for jobs, including pass/fail.. It's not quite a list, but if you know graphite, you can probably get the data you wan09:32
Kiallt09:32
Kialle.g. for the first job I found ;) http://graphite.openstack.org/render/?width=586&height=308&_salt=1434706376.002&target=stats_counts.zuul.pipeline.check.job.check-devstack-dsvm-cells.ABORTED&target=stats_counts.zuul.pipeline.check.job.check-devstack-dsvm-cells.FAILURE&target=stats_counts.zuul.pipeline.check.job.check-devstack-dsvm-cells.SUCCESS09:33
*** amotoki_ has quit IRC09:34
marzif_any core can help me moving this forward please https://review.openstack.org/#/c/193125/09:35
*** coolsvap|away is now known as coolsvap09:37
*** bexelbie has joined #openstack-infra09:38
*** romcheg has joined #openstack-infra09:39
*** jlanoux_ has joined #openstack-infra09:40
*** abregman has quit IRC09:41
*** abregman has joined #openstack-infra09:41
*** jasondotstar has joined #openstack-infra09:41
*** jlanoux has quit IRC09:42
*** hashar has quit IRC09:45
*** _nadya_ has joined #openstack-infra09:45
flip214Kiall: thanks. I do not know it now, but perhaps I will...09:47
*** hashar has joined #openstack-infra09:48
*** abregman has quit IRC09:49
*** Kennan2 is now known as Kennan09:49
*** e0ne is now known as e0ne_09:52
*** e0ne_ is now known as e0ne09:54
flip214Kiall: thanks, could make it working09:59
*** amotoki_ has joined #openstack-infra10:07
*** ddieterly has joined #openstack-infra10:08
*** dims has joined #openstack-infra10:09
*** Hal has joined #openstack-infra10:09
*** Hal is now known as Guest1358610:10
*** coolsvap is now known as coolsvap|away10:11
*** ddieterly has quit IRC10:13
*** markvoelker has joined #openstack-infra10:13
*** markvoelker has quit IRC10:18
Odd_Blokesdague: Did you see my comment on https://review.openstack.org/#/c/191081/3/zuul/layout.yaml ?10:19
*** sc` has quit IRC10:21
*** Alexandra_ has joined #openstack-infra10:24
*** kushal has joined #openstack-infra10:24
marzif_sdague, AJaeger I really need help on https://review.openstack.org/#/c/193125/ I'm blocked...10:25
flip214Kiall: I tried to login to graphite now, hoping that I can store the graph (or export it as URL).10:25
flip214But all I got was "Graphite encountered an unexpected error while handling your request."10:25
sdagueOdd_Bloke: so what's the test failure scenario?10:26
flip214http://paste.openstack.org/show/304478/ is the complete error message10:26
Odd_Blokesdague: If coverage drops below 90%.10:27
flip214How can I get an URL that shows me that data?10:27
sdaguethat still seems weird to gate on, because that means some merge put you accross a threshold and it's going to be really hard to debug10:28
AJaegermarzif_: I'm quite busy with other work today, I'll see what I can do for that one later.10:28
sdaguewe're enforcing clean check, you can't get into the gate unless this test has passed10:28
marzif_AJaeger,  appreciate your answer10:28
AJaegermarzif_: just a quick question: Do you want to import some parts of your repository into the new repo together with history? In that case use the "upstream" keyword and setup a new repo using "git filter".10:28
Odd_Blokesdague: What might have changed between check and gate?  HEAD of the branch we're merging in to?10:29
sdagueOdd_Bloke: yep10:29
*** aarefiev has quit IRC10:29
marzif_AJaeger,  yes I want to do that - this is the option I was thinking about:  git filter-branch --prune-empty --subdirectory-filter horizon_web_ui/ master10:30
sdagueand, it might not even be in head yet. It could be multiple things in flight10:30
marzif_AJaeger,  I'll check your solution10:30
sdagueso you push 3 patches into gate they collide and bounce off each other, and no one understands why10:30
sdagueit just seems like a weird gate job. If you really want it, so be it, but I'd recommend against it10:30
Odd_Blokesdague: I'll chat with harlowja (PING), and see what he thinks.10:31
AJaegermarzif_: you only have one chance for that - now. Once your patch is in, we cannot import, this needs to be setup as part of your patch.10:31
AJaegermarzif_: check the infra manual and how oslo does it.10:32
marzif_AJaeger, ok10:32
flip214thanks, got it10:38
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: Disable several tempest jobs for Manila stable/juno branch  https://review.openstack.org/19349110:39
*** afazekas has quit IRC10:44
*** hashar has quit IRC10:45
openstackgerritDavide Guerri proposed openstack-infra/shade: Add Neutron/Nova Floating IP list/search/get  https://review.openstack.org/18349610:47
openstackgerritDavide Guerri proposed openstack-infra/shade: Add Neutron/Nova Floating IP create (allocate to project)  https://review.openstack.org/18355510:47
*** samueldmq has quit IRC10:51
openstackgerritFausto Marzi proposed openstack-infra/project-config: Splitting stackforge/freezer on multiple repos  https://review.openstack.org/19312510:54
*** dims has quit IRC10:57
*** rguillebert has joined #openstack-infra11:00
*** pc_m has joined #openstack-infra11:00
*** Guest13586 has quit IRC11:01
*** sc` has joined #openstack-infra11:01
*** samuel has joined #openstack-infra11:02
*** samuel has quit IRC11:02
*** samuel has joined #openstack-infra11:03
*** samuel has quit IRC11:03
*** samueldmq has joined #openstack-infra11:04
*** wuhg has quit IRC11:04
*** AzherKhan has quit IRC11:04
samueldmqmorning :)11:05
*** dims has joined #openstack-infra11:06
*** afazekas has joined #openstack-infra11:07
*** ddieterly has joined #openstack-infra11:09
*** aarefiev has joined #openstack-infra11:11
*** ddieterly has quit IRC11:14
*** Hal has joined #openstack-infra11:18
*** Hal is now known as Guest7035311:19
*** e0ne is now known as e0ne_11:19
*** ala_ has joined #openstack-infra11:24
marzif_AJaeger, I understand your are busy, so please reply at your convenience: I'm not sure why I need to track upstream to do that11:24
marzif_here I wrote some instruction int he dashboard field on how to possible to that11:24
marzif_https://blueprints.launchpad.net/freezer/+spec/main-repo-split11:24
AJaegermarzif_: you're not tracking upstream11:24
marzif_I'm not sure I need to do that11:24
AJaegerupstream keyword means: Seed this directory from the other one.11:25
AJaegerAlternative is: You get an empty repository and push everything through gerrti - and loose your revision history11:25
AJaegerthe usual work flow is: use git filter to create a new repository with the initial content for your new repo.11:26
AJaegerMake that new repo public somewhere and reference it via the upstream keyword11:26
*** otter768 has joined #openstack-infra11:26
AJaegerOnce we approve the change, the "upstream" repo gets imported into your new repo11:26
*** Alexandra_ has quit IRC11:27
*** Ala has quit IRC11:27
marzif_I'll do that because it's the usual way and I don't want to change it, but I think the steps are not necessary, without losing the history, I'll follow your indication anyway11:28
AJaegermarzif_: It's the best way doing it with our infrastructure.11:29
*** markvoelker has joined #openstack-infra11:29
marzif_AJaeger, ok : )11:29
*** e0ne_ has quit IRC11:29
*** koolhead17 has quit IRC11:31
*** mwagner_afk has quit IRC11:31
*** otter768 has quit IRC11:31
*** markvoelker has quit IRC11:34
tchayposdague: before reading https://review.openstack.org/#/c/192253/, I’d been expecting to start with talking about what data we need to collect; what’s already in graphite, what we can easily put there by tweaking zuul, what we’re going to have to collect from elsewhere. Starting the conversation with what technology we’re going to used to present the11:42
tchaypodata surprised me. In hindsight it makes sense - we don’t want to start throwing data into a system we’ll be throwing away.11:43
*** jasondotstar has quit IRC11:43
*** kushal has quit IRC11:44
tchaypomarzif_: AJaeger: The first time I saw the “upstream” keyword there I was surprised too. It made me think of a git upstream - something that gets constantly tracked and pulled from. It took me a while to understand that in this context it’s something we read from once, but afterwards never look at again11:44
marzif_AJaeger, I've replied to your comments https://review.openstack.org/#/c/193125/ if we could use the openstack name space I think would be better, the project now is growing, it will save us to do similar work in the future11:45
marzif_tchaypo, yes, agreed, git-upstream was written by the guy in front of my desk : )11:45
*** abregman has joined #openstack-infra11:46
sdaguetchaypo: yeh, so, the implementation steps hopefully provide some of that road map.11:46
sdaguebecause I expect that what we need is going to get discovered over time as things get implemented11:47
tchayposdague: so I don’t have any particular thoughts on what we want to use. I assume we’d want something python if we can, but I hadn’t really thought past that.11:47
tchaypoI assume we’ll find over time that the data we thought was interesting isn’t actually the thing we end up referring to, so something that can be easily tweaked and drilled down into would be nice11:47
*** romcheg has quit IRC11:48
*** erlon has joined #openstack-infra11:50
*** boris-42 has quit IRC11:52
*** koolhead17 has joined #openstack-infra11:53
*** maishsk_afk has quit IRC11:54
*** mrunge has quit IRC11:54
sdaguetchaypo: yeh, I think it most cases we'll be able to get more data put back into graphite11:54
*** maishsk_afk has joined #openstack-infra11:56
*** deepakcs has quit IRC11:56
*** weshay has joined #openstack-infra11:58
AJaegermarzif_: openstack namespace is a TC decision - shouldn't be hard with the new big tent model. Best talk later with jeblair or with sdague about that.11:59
*** marcusvrn has joined #openstack-infra11:59
AJaegertchaypo: yeah, upstream is misnamed ;(11:59
AJaegertchaypo: that's why we remove these semi-automatically after import ;)11:59
*** asahlin_away has quit IRC12:00
tchayposdague: I’m assuming we’re not going to decide these things before, or during, next week’s meeting - but hopefully we’ll identify who is interested in driving the choices12:00
marzif_AJaeger, ok, ty12:01
*** bknudson has quit IRC12:01
*** asahlin has joined #openstack-infra12:01
*** marcusvrn has quit IRC12:01
sdaguetchaypo: yeh, honestly, if I was just hacking on this myself I'd have started coding already, but it's pretty clear I'm not going to have a huge amount of coding cycles for this, so I'm just trying to curate the needs12:03
*** marcusvrn has joined #openstack-infra12:04
tchaypothanks :)12:04
*** dprince has joined #openstack-infra12:05
*** markvoelker has joined #openstack-infra12:05
*** BharatK has quit IRC12:05
openstackgerritDavide Guerri proposed openstack-infra/shade: Add Neutron/Nova Floating IP create (allocate to project)  https://review.openstack.org/18355512:07
*** e0ne has joined #openstack-infra12:08
*** radez_g0n3 is now known as radez12:09
*** ddieterly has joined #openstack-infra12:10
*** romcheg has joined #openstack-infra12:10
*** dtantsur|brb is now known as dtantsur12:11
*** coolsvap|away is now known as coolsvap12:12
*** ddieterly has quit IRC12:14
*** laughterwym has joined #openstack-infra12:17
openstackgerrityolanda.robla proposed openstack-infra/nodepool: Different queues for slow tasks.  https://review.openstack.org/17078812:19
*** yamamoto has quit IRC12:19
*** tlian has joined #openstack-infra12:20
*** coolsvap is now known as coolsvap|away12:22
*** bknudson has joined #openstack-infra12:25
*** unicell1 has quit IRC12:26
*** kgiusti has joined #openstack-infra12:26
*** EricGonczer_ has joined #openstack-infra12:26
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: New oslo library - oslo.service  https://review.openstack.org/19288612:26
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: New oslo library - oslo.service  https://review.openstack.org/19288612:26
*** trown has joined #openstack-infra12:28
*** unicell has joined #openstack-infra12:29
AJaegerjeblair: could you look at https://review.openstack.org/#/c/193392/ and https://review.openstack.org/#/c/191131/ and talk with the Ironic team, please? We might need some kind of "official" message for usage of stackforge vs. openstack for reference12:29
*** rossella_s has quit IRC12:30
*** unicell has quit IRC12:30
*** rossella_s has joined #openstack-infra12:31
*** rlandy has joined #openstack-infra12:33
*** coolsvap|away is now known as coolsvap12:34
*** EricGonczer_ has quit IRC12:35
*** xyang1 has joined #openstack-infra12:36
*** dpaterson has joined #openstack-infra12:38
*** pelix has joined #openstack-infra12:39
trownAJaeger: thanks for the feedback on ahc-tools12:39
dtantsurjeblair, seconding the request :) we might have a wrong idea what stackforge is for12:40
trownAJaeger: I understand that with the "big tent" that we no longer strictly need to have projects incubate in stackforge12:40
dtantsurI'm still not sure however, how to prove usefullness of the project without it being baked somewhere12:40
AJaegerdtantsur, trown: Times change ;)12:40
trownAJaeger: what about immature projects that do actually need incubation?12:40
*** woodster_ has joined #openstack-infra12:40
trownAJaeger: I would consider ahc-tools very immature at this point12:40
AJaegertrown: If it'S not successful, you can make it read-only, independent where it is.12:41
*** Longgeek has quit IRC12:41
*** bswartz has quit IRC12:41
fungidtantsur: yeah, the idea with the big tent is that teams get recognized by the tc, and as long as they stick with the scope they've defined they can have as many repos as they want in openstack. there's no reason for it to wait for usability or progress before making it openstack, as long as it's being handled by the team the tc has already recognized12:41
trownAJaeger: I guess my question is, what is the purpose of stackforge namespace at this point12:42
fungitrown: very little12:42
dtantsurfungi, here is the problem: of the whole team only I know this project12:42
AJaegertrown: With the big tent, it'S more about the 4 opens and being part of OpenStack for a team than mature repositories.12:42
fungitrown: i and a number of others would like to just get rid of it12:42
*** Longgeek has joined #openstack-infra12:42
trownfungi: ah that is useful info12:43
*** jasondotstar has joined #openstack-infra12:43
dtantsurwow, never thought about it12:43
fungitrown: i wish we didn't have these git namespaces at all, but they started because we needed to map repo names to orgs we controlled in github for replication purposes12:43
AJaegertrown, dtantsur: For your project - and that'S why I asked - I see no need to be part of stackforge. Only costs and pain for fungi when you move from stackforge to openstack ;)12:43
trownAJaeger: I like your optimism :)12:43
dtantsurlol12:43
dtantsurAJaeger, fungi, that all makes a lot of sense to me, thanks! But I believe someone has to write a lot of long ML posts explaining this new position12:44
fungithe namespaces themselves accumulated some unwarranted meaning, but they're entirely cosmetic (and a serious amount of work to change after the repo is created)12:44
marzif_AJaeger,  fungi what would be needed to do to migratee a repo from stackforge to openstack namespace?12:44
AJaegertrown: It's a shift in mindset: Moving from "mature projects in openstack/, incubation in stackforge" to "repos can mature - or fail - in openstack"12:45
fungimarzif_: we take down all our developer services for a maintenance window and do http://docs.openstack.org/infra/system-config/gerrit.html#renaming-a-project12:45
marzif_as I need to split components, would be probably helpful do it now, and do the job only once...12:45
marzif_ah ok fungi thanks12:45
AJaegerfungi: marzif_'s background: https://review.openstack.org/193125 which proposes a split of a repo in stackforge...12:46
fungiso gerrit, zuul et cetera are offline for 30 minutes or more12:46
*** dims has quit IRC12:46
*** dims has joined #openstack-infra12:46
AJaegerfungi: seems jeblair needs to put on his marketing head for some emails or ask others in the TC to help out ;)12:47
fungiand basically the 2k+ developers we have working on openstack can't push changes or get them tested until we finish the renaming work for a single project12:47
dtantsurmarzif_, I feel sorry for you, splitting a project can be painful :) doing the same right now12:47
fungiAJaeger: yeah, i'm hoping https://review.openstack.org/192016 is able to make some headway, after further revision12:48
marzif_dtantsur, lol doing it once, it's part of the game, but doing the same twice...12:48
*** mrunge has joined #openstack-infra12:48
trownAJaeger, fungi, I am still a little unclear on the criteria for putting a repo in the openstack namespace12:48
*** aysyanne has joined #openstack-infra12:49
trownAJaeger: fungi, I get that it is not "mature stable project", I just dont get what it is instead12:49
marzif_trown, I think it's still in discussion probably...12:49
trownAJaeger: fungi, ie is it everything that would have got accepted to stackforge before?12:50
*** sarob has joined #openstack-infra12:50
fungitrown: if it's part of a tc recognized project, it's simply necessary that the project says it'll be taking on governance of that repo (either directly or through some auxiliary subteam)12:50
*** dizquierdo has joined #openstack-infra12:50
fungitrown: if it's a new project rather than joining an existing project, http://governance.openstack.org/reference/new-projects-requirements.html are the guidelines12:51
*** jaypipes has quit IRC12:51
fungiand i think even that document makes some unnecessary assumptions about "maturity" of a team's work12:52
trownfungi: perfect, that answers my question quite well12:52
fungii'll see if i can come up with a better wording to propose to the tc12:52
dtantsurfungi, that all sounds good. I wonder what happens if a PTL does not approve including of subproject... would it be a use case for stackforge?12:52
*** ldnunes has joined #openstack-infra12:53
openstackgerritDavide Guerri proposed openstack-infra/shade: Add Neutron/Nova Floating IP create (allocate to project)  https://review.openstack.org/18355512:53
fungidtantsur: more likely the case for it to be a separate project. a good example is the searchlight project which just spun out of an effort in glance12:53
fungiit's closely related to glance, but since it has its own team and they're already working together quite effectively, they simply because a new tc recognized project (last week or the week before i think?)12:54
dtantsurhmm, interesting12:54
fungis/because/became/12:54
dtantsurok folks, thanks for clarifying, much appreciated!12:55
*** sarob has quit IRC12:55
krtaylorinteresting discussion, I'll also throw in that a stackforge project doesn't have to have the TC blessing to start, I always viewed it as a place to start forming, then get TC recognized, then promotion12:55
*** dpaterson has quit IRC12:55
fungidtantsur: the resolution to make searchlight a new project is at https://review.openstack.org/18801412:56
fungithey're one of many, but i use them as an example because they didn't follow the pattern of "create a repo in stackforge, apply to the tc, move the repo from stackforge to openstack"12:56
*** pblaho has joined #openstack-infra12:56
*** yamamoto has joined #openstack-infra12:57
*** unicell has joined #openstack-infra12:57
dtantsurI see12:57
*** yamamoto has quit IRC12:57
fungiinstead they did "talk to an existing project about inclusion, apply to the tc, create a repo in openstack"12:57
*** yamamoto has joined #openstack-infra12:57
*** Longgeek has quit IRC12:57
trownfungi: thanks again, all very helpful information, I will go that route and talk to Ironic first12:58
trownand see what happens from there12:58
fungiand if glance had agreed instead, the process for searchlight would have been simply "talk to an existing project about inclusion, create a repo in openstack"12:59
AJaegertrown: thanks!12:59
trownAJaeger: thank you for helping clear up the confusion12:59
AJaegerfungi: Thanks for jumping in, here!12:59
*** Longgeek has joined #openstack-infra12:59
*** coolsvap is now known as coolsvap|away12:59
*** unicell has quit IRC13:00
fungibut i like the timeline on that tc motion. searchlight proposed a new project to the tc on june 3, and within 6 days it was approved as a new openstack project. clearly the tc is getting a lot faster at this now13:02
*** dboik has joined #openstack-infra13:03
*** eharney has joined #openstack-infra13:03
*** romcheg has quit IRC13:04
fungithe motion is all of ~40 lines of prose and half a dozen bits of metadata13:04
*** sdake has joined #openstack-infra13:05
*** julim has joined #openstack-infra13:05
*** HenryG has quit IRC13:05
*** dboik_ has joined #openstack-infra13:05
*** HenryG has joined #openstack-infra13:05
*** HenryG has quit IRC13:06
*** rfolco has joined #openstack-infra13:06
fungiand now they can create however many repos they need to achieve their stated mission13:06
*** sdake_ has joined #openstack-infra13:07
*** julim has quit IRC13:07
*** HenryG has joined #openstack-infra13:07
*** soren has joined #openstack-infra13:08
*** julim has joined #openstack-infra13:08
*** dboik has quit IRC13:08
*** devvesa has quit IRC13:08
*** claudiub has quit IRC13:08
*** mmedvede has quit IRC13:09
mordredfungi: ok. shade 0.8.1 is on nodepool13:09
fungimordred: excellent. should we restart it before image update time?13:09
mordredfungi: I'd like to restart nodepool to pick up the change - that ok with you?13:09
mordredfungi: yah13:09
fungigo for it. can you give me about 5 minutes though? i'm finishing up a check for alient nodes to see if i can gauge how well the new cleanup is working13:10
*** laughterwym has quit IRC13:10
fungis/alient/alien/13:10
*** ddieterly has joined #openstack-infra13:10
*** sdake has quit IRC13:10
*** laughterwym has joined #openstack-infra13:10
mordredsure!13:11
fungimainly just a spot check to see if i can find (via the old traditional comparison method) any which have somehow slipped through the new metadata-based autocleanup13:11
samueldmqmordred: nice, I starte looking at shade yesterday, left a review on one of those patches keystone-related13:12
mordredfungi: btw - "sudo -H nodepool shade-inventory --yaml --list" should give you a lovely yaml output of all of the nodes in nodepool, if you ever want such a thing13:12
mordredsamueldmq: woot! thanks13:12
*** dustins has joined #openstack-infra13:13
*** baoli has joined #openstack-infra13:14
fungimordred: that's excellent--new nodepool cli subcommands to play with13:14
fungioh, wait, it's a separate shade cli. still cool13:14
fungiassuming you meant -u nodepool?13:15
*** ddieterly has quit IRC13:15
fungirather -H -u nodepool13:15
samueldmqmordred: that's still a small project, and I guess it's intended to be simple :)13:15
openstackgerritMerged openstack-dev/pbr: Strip markers from test requirements.  https://review.openstack.org/19337213:15
samueldmqmordred: it will be fun to take a deeper look on it13:15
*** maishsk_afk has quit IRC13:16
mordredfungi: yes. I did mean -H -u nodepool13:16
samueldmqmordred: as I find some time between dynamic policies tasks ;)13:16
fungiindeed, `sudo -H -u nodepool shade-inventory --yaml --list` works. just tried it13:16
sdaguesamueldmq: it will be more fun if we can make it obsolete by fixing openstack13:16
*** baoli_ has joined #openstack-infra13:16
mordredyup13:16
mordredthat's the goal13:16
mordredbut hopefully it'll be a clear target to aim for13:17
fungisamueldmq: sdague: the smaller shade gets, the more successful openstack is becoming at achieving consistency and interoperability13:17
samueldmqsdague: ahha yes I agree13:17
samueldmqsdague: mordred fungi that's great! :)13:17
*** yamamoto has quit IRC13:17
*** koolhead17 has quit IRC13:18
samueldmqmordred: https://github.com/openstack-infra/shade/blob/master/shade/__init__.py#L202-L20713:18
samueldmqmordred: this is a great description on what it is different fro ma simple OpenStack sdk13:19
AJaegersdague: should we add a link to governance or one of its subpages on docs.o.o ? I haven't found the proper text and place yet.13:19
samueldmqmordred: as we were discussing a couple of days ago, iirc13:19
*** bswartz has joined #openstack-infra13:19
*** baoli has quit IRC13:19
*** romcheg has joined #openstack-infra13:19
AJaegersdague: I just read your comment on the stackforge -> openstack TC proposal13:20
*** sdake has joined #openstack-infra13:20
openstackgerritWayne Warren proposed openstack-infra/infra-specs: Jenkins Job Builder 2.0.0 API Changes/Rewrite  https://review.openstack.org/18254213:21
openstackgerritWayne Warren proposed openstack-infra/infra-specs: Jenkins Job Builder 2.0.0 API Changes/Rewrite  https://review.openstack.org/18254213:21
*** weshay is now known as weshay_mtg13:22
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Convert ironicclient node.get() call to Task  https://review.openstack.org/19353513:23
*** achanda has joined #openstack-infra13:23
*** sdake_ has quit IRC13:24
*** mriedem has joined #openstack-infra13:24
mordreddguerri`, Shrews: oops. error in the get_ipv4 code13:25
EmilienMmordred:,AJaeger: good morning here! I think I did wrong yesterday in trying to enable voting for puppet4. Here is a fix: https://review.openstack.org/#/c/193379/13:25
*** zz_dimtruck is now known as dimtruck13:25
sdagueAJaeger: that would probably be a good thing to do13:25
sdagueAJaeger: I don't know the best place13:25
AJaegersdague: let me do a patch and see what others think...13:26
AJaegerit's not the first time it came up but I haven't found the perfect place - so time for a non-perfect one ;)13:26
*** jaypipes has joined #openstack-infra13:26
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: Add several jobs for Manila periodic pipeline  https://review.openstack.org/19353813:26
*** dguerri` is now known as dguerri13:26
sdagueI also kind of feel that - http://governance.openstack.org/reference/projects/index.html should at least have a summary of the projects in question, that's most a weird wall of text that means nothing to people that don't already know what it means13:26
AJaegersdague: yeah13:26
openstackgerritMonty Taylor proposed openstack-infra/shade: Catch all exceptions around port for ip finding  https://review.openstack.org/19354013:26
fungimordred: jeblair: interestingly i have 10 alien nodes showing in dfw, 3 in iad and 2 in ord (in addition to the 4 in hpcloud which i think are undeletable, "locked" according to novaclient)13:27
Shrewsmordred: fix it fix it fix it13:27
mordredfungi, Shrews: ^^ I need to land that patch now13:27
mordredbecause inventory is broken for rax13:27
*** otter768 has joined #openstack-infra13:27
dguerrimordred: looking13:27
*** yamamoto has joined #openstack-infra13:27
mordreddguerri: then, I think after we cut a bugfix - we should probably go add another test to highlight that issue, if you have a sec13:28
fungimordred: any reason not to build up a list of known exceptions there? is it safe to just treat any exception that might throw equally?13:28
mordredfungi: it's safe for now, since that's a function where if one service doesn't work we're just going to try another one13:29
dguerrican't open https://review.openstack.org ... hope it's just me13:29
*** bcourt has joined #openstack-infra13:29
fungidguerri: i assume it's just you because it's working for me13:29
*** devvesa has joined #openstack-infra13:29
dguerrifungi: tnx13:29
mordreddguerri: hrm. something is off in that code in general - I do not have a good accessIPv4 on any of my clouds13:29
*** abregman has quit IRC13:30
mordreddguerri: I'm going to put in some local tracing and let you know what's up13:30
dguerrimordred: ++13:30
*** abregman has joined #openstack-infra13:30
fungimordred: should i un-approve that exception change?13:30
dguerrimordred: that's quite odd, we tested it quite a bit before merging13:30
*** peristeri has joined #openstack-infra13:30
fungior are you planning to submit a different change for the rest of whatever you find rather than evolving that one?13:30
mordredfungi: exception change is still great13:31
mordredit's actually _totally_ broken13:31
fungicool, i'll leave it in flight13:31
Shrews:(13:31
mordredthis other thing is a specific metadata thing13:31
*** dannywilson has joined #openstack-infra13:32
*** otter768 has quit IRC13:32
mordredShrews, dguerri: we're missing a layer - you're trying to map a server port to a network - but a server port is going to be attached to a subnet which will be attached to a network13:34
AJaegersdague: what do you think of https://review.openstack.org/193547 for the governance change on docs.o.o?13:35
*** trown has left #openstack-infra13:35
*** puranamr has joined #openstack-infra13:36
*** mmedvede has joined #openstack-infra13:37
AJaegersdague, fungi: could either of you unblock kolla, please?  https://review.openstack.org/19336913:37
*** maishsk_afk has joined #openstack-infra13:38
sdakewould be much appreciated :)13:38
sdague+A13:38
fungilgtm13:38
fungiwe double-approved it, for double goodness i guess13:38
AJaegersdague, fungi : Either not both ;)13:38
AJaegerfungi: indeed - thanks13:39
openstackgerritAntoine Musso proposed openstack/diskimage-builder: doc: migrate from README.rst to Sphinx  https://review.openstack.org/19305113:39
*** tvanderwiel has joined #openstack-infra13:39
*** dguerri is now known as dguerri`13:40
*** asselin has quit IRC13:40
*** ddieterly has joined #openstack-infra13:41
*** weshay_mtg is now known as weshay13:41
*** ghostpl has joined #openstack-infra13:43
*** ddieterly has quit IRC13:44
*** prad has joined #openstack-infra13:44
*** dguerri` is now known as dguerri13:45
*** dansmith is now known as superdan13:45
*** dannywilson has quit IRC13:46
*** ddieterly has joined #openstack-infra13:46
*** yolanda has quit IRC13:46
*** dannywilson has joined #openstack-infra13:46
*** yamamoto_ has joined #openstack-infra13:47
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Convert ironicclient node.update() call to Task  https://review.openstack.org/19355313:48
*** yolanda has joined #openstack-infra13:48
mordredShrews, dguerri: ok - may have been pebkac13:48
ShrewsTheJulia: final two reviews that convert ironic calls to Task: https://review.openstack.org/193535 and https://review.openstack.org/19355313:48
*** nihilifer has quit IRC13:48
dguerrimordred: please tell me it is :)13:49
mordredShrews, dguerri: yup. sorry, there is a field called "accessIPv4" in the nova metadata that isn't actually what we use and isn't actually useful13:49
mordredI was looking at the wrong thing - it all works13:49
fungiAJaeger: dtantsur: krtaylor: https://review.openstack.org/193550 is my proposal to make the new project requirements look less like a suggestion to incubate projects in stackforge first13:49
dguerrithank god13:50
mordredyah13:50
dguerrio/13:50
* dtantsur is looking13:50
Shrewsmordred: yay! that exception patch is still needed though. we should have caught that13:50
dguerri(other arm is doing an italian thing I can't explain)13:50
mordreddguerri: hahahaha13:50
dguerri;)13:50
*** jaypipes is now known as leakypipes13:50
mordredI almost think we shoudl fill in accessIPv4 and accessIPv6 since they're blank everywhere and we actuallyknow the answer13:51
mordredsdague: is nova supposed to be providing those back?13:51
AJaegerfungi, thanks!13:51
dguerrirght13:51
*** yamamoto has quit IRC13:51
*** ddieterly has quit IRC13:52
*** maishsk_afk has quit IRC13:52
mordredah - so, it IS filled in on rackspace, but nobody else has bothered13:53
* mordred hands rackspace a gold star13:53
dguerrierheh13:53
Shrewsjroll: congrats!13:53
mordreddguerri: we could add a  piece to the logic "if server['accessIPv4']:" to bypass all the rest of the stuff if the cloud is nice enough to have given us the data13:53
Shrewsthat sounds worthwhile13:54
dguerriyup13:54
mordreddguerri: it would actually mean that our fallback of "hrm, not sure, let's see if it's a public address might be removable"13:54
mordreddguerri: it would actually mean that our fallback of "hrm, not sure, let's see if it's a public address" might be removable13:54
mordredbad quoting13:54
* mordred test drives that thought real quick13:54
*** amitgandhinz has joined #openstack-infra13:54
openstackgerritMerged openstack-infra/project-config: Revert "Make kolla functional job voting"  https://review.openstack.org/19336913:54
*** ddieterly has joined #openstack-infra13:54
jrollShrews: uh oh, what'd I do?13:54
dimsfungi: please add oslo-core to oslo-reports-core and oslo-reports-release gerrit groups when you get a chance13:55
pabelangermorning13:55
Shrewsjroll: you get mordred's gold star by proxy13:55
jroll\o/13:55
jrollmordred: so afaik nova is supposed to provide those, but they are able to be set/modified by the user (or other things?) and are not to be trusted13:55
sdaguemordred: so the upstream code feels like that should be thre13:56
dguerrimordred: is accessIPv4 rax only?13:56
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: Add several jobs for Manila periodic pipeline  https://review.openstack.org/19353813:56
mordredjroll: what other things would be ... wait ... aroo?13:56
*** pblaho has quit IRC13:56
mordreddguerri: no, they're openstack constructs13:56
mordreddguerri: it just seems liek rax is the only cloud who is filing them in13:57
*** mwagner_afk has joined #openstack-infra13:57
mordredexcept now jroll tells me that somethign might cause them to lie13:57
fungidims: done13:57
*** dims is now known as dimsum__13:57
mordredjroll: honestly, if a user overwrites that value with something else somehow, they're on their own I cannot be responsible for insanity13:57
dimsum__fungi: thanks!13:57
jrollmordred: indeed. I could just imagine neutron or something doing insane things and setting it. idk.13:58
*** hashar has joined #openstack-infra13:58
jrollmordred: but you're probably fine to trust it13:58
mordredjroll: it does deflate a little bit of my rage at rackspace networking though13:58
jroll\o/13:58
fungimordred: so, we're down to ~15 minutes before image updates. are we safe to go ahead and restart nodepoold now?13:58
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Pin to a specific fedora mirror  https://review.openstack.org/19356013:58
mordredfungi: yes!13:58
jrollmordred: feel free to continue sending rage this way13:59
fungimordred: okay, i'll do it unless you wanted to13:59
fungilooks like nodepool is keeping pace with zuul demand for the moment13:59
*** koolhead17 has joined #openstack-infra13:59
*** ParsectiX has quit IRC13:59
mordredfungi: sorry - I got distracted by the other shade bug13:59
fungithough until we get jeblair's 193368 tested out and applied, we're going to see nodepool lose track of zuul demand while glance uploads happen (unless latest shade has broken our uploads today!)14:00
dguerrimordred: not 100% sure what you have in mind, I would keep the public IP check anyway14:00
fungimordred: okay, nodepoold has restarted now14:01
dguerrimordred: we can just skip all the code if we have the accessIPv4 available14:01
fungii'm tailing the debug log to see if there is any obvious crazy14:01
fungitripleo cloud seems to be pitching us 500 errors14:02
*** dannywilson has quit IRC14:02
Shrewsmordred: fungi: the exception change did not merge. does that break nodepool still?14:02
fungii'm assuming that's just business as usual there14:02
*** claudiub has joined #openstack-infra14:02
fungiShrews: oh, not sure. i bet we need a new release tagged with that fix in place?14:02
*** morgan has quit IRC14:02
fungimordred: ^?14:03
*** morgan has joined #openstack-infra14:03
fungilooks like the patch is 8 minutes out from completing its devstack functional jobs14:03
fungiin check14:03
*** morgan is now known as Guest9312614:03
fungiand then it will i guess need another half hour in the gate, so not going to be in time for today's image updates14:04
fungishould i downgrade shade and restart nodepoold again?14:04
fungiif so, i have about 10 minutes to do that14:04
mordredShrews: no, it does not14:05
mordredthe exception change won't affect nodepool - that's just a break on inventory14:06
Shrewsmordred: ok. the approval urgency led me to believe otherwise14:06
*** morganfainberg is now known as morgan-devserver14:06
fungiexcellent. then i'll just leave this chugging14:06
openstackgerritMerged openstack/requirements: Add glance_store to projects.txt  https://review.openstack.org/18268114:07
mordredShrews: sorry - it was just a bug that would break ansible inventory for anyone with rackspace, and I know ansible tower has started using openstack inventory module, so I didn't want to leave a known hard break (you get no inventory back) in the wild14:07
*** tonytan4ever has joined #openstack-infra14:07
fungithe tripleo cloud seems really unhappy. the exceptions look like they're related to floating ip allocation. maybe we've leaked some?14:07
*** morgan-devserver is now known as CaptainMorgan14:08
AJaegersdague: governance side can soon be found on docs.openstack.org home page - patch has been approved14:08
*** ihrachyshka has quit IRC14:09
*** ajmiller has joined #openstack-infra14:10
*** puranamr has quit IRC14:10
*** puranamr has joined #openstack-infra14:11
sdagueAJaeger: cool14:12
*** sigmavirus24_awa is now known as sigmavirus2414:12
*** krtaylor has quit IRC14:15
openstackgerritMerged openstack-infra/elastic-recheck: Add query for neutron update port bug 1465407  https://review.openstack.org/19198114:15
openstackbug 1465407 in neutron "StaleDataError: UPDATE statement on table 'ports' expected to update 1 row(s); 0 were matched." [High,In progress] https://launchpad.net/bugs/1465407 - Assigned to Kevin Benton (kevinbenton)14:15
openstackgerritAntoine Musso proposed openstack-infra/project-config: nodepool: describe nodepool-base env variables  https://review.openstack.org/19356614:16
openstackgerritAntoine Musso proposed openstack-infra/project-config: nodepool: nodepool-base can now skip unbound conf  https://review.openstack.org/19356714:16
hasharI should really just create my own element :-D14:16
*** asselin has joined #openstack-infra14:16
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Catch all exceptions around port for ip finding  https://review.openstack.org/19354014:16
Shrewsmordred: ^^^^ fixed the pep8 error for you14:16
*** puranamr has quit IRC14:16
*** yfried_ has quit IRC14:20
*** zz_jgrimm is now known as jgrimm14:20
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: remove query for bug 1342380  https://review.openstack.org/19356814:21
openstackbug 1353939 in OpenStack Compute (nova) "duplicate for #1342380 Rescue fails with 'Failed to terminate process: Device or resource busy' in the n-cpu log" [High,Fix committed] https://launchpad.net/bugs/1353939 - Assigned to Matt Riedemann (mriedem)14:21
openstackgerritMonty Taylor proposed openstack-infra/shade: Use accessIPv4 if it's there  https://review.openstack.org/19356914:21
mordredShrews: cool. thanks! and there's the patch to use accessIPv4 if it exists14:21
mordreddguerri: ^^14:21
*** maishsk_afk has joined #openstack-infra14:22
*** stevemar has joined #openstack-infra14:24
derekh fungi RE tripleo floating ip's looks like your bumping up against the floating IP quota, should we delete the unsed floating IP's?14:24
derekhdprince: ^14:24
*** soren has quit IRC14:25
derekhWe also have 8 VM's not being used so could do with some eyes on this https://review.openstack.org/#/c/187675/214:25
*** fallenpegasus has joined #openstack-infra14:25
fungiderekh: we've seen leaks in other providers in the past (exhibited as fips which are not associated with any instance)14:26
fungiseems to happen when nodepool requests a fip deletion and for whatever reason it doesn't happen (but possibly reports that it did)14:26
*** leopoldj has joined #openstack-infra14:27
*** hdd has joined #openstack-infra14:27
*** r-daneel has joined #openstack-infra14:27
openstackgerritMerged openstack-infra/elastic-recheck: remove query for bug 1342380  https://review.openstack.org/19356814:28
openstackbug 1353939 in OpenStack Compute (nova) "duplicate for #1342380 Rescue fails with 'Failed to terminate process: Device or resource busy' in the n-cpu log" [High,Fix committed] https://launchpad.net/bugs/1353939 - Assigned to Matt Riedemann (mriedem)14:28
dprincederekh: we just reset all the floating ips on MOnday I think14:28
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: add query for keystone wsgi restart issue  https://review.openstack.org/19357114:28
dprincederekh: so any leak we've got would be a fast one...14:28
derekhfungi dprince : ok, so it may have been like this since the probelms we had on monday, currently there is 111 FIP's in the tenants with 27 associated with a VM14:28
*** stevemar is now known as stevedoor14:29
*** alaski is now known as lascii14:29
jeblairAJaeger, fungi: i think all the questions about stackforge got answered?14:30
derekhfungi: dprince so if nodepool doesn't know about them I guess we can delete them and see if they keep leaking14:30
derekhbrb14:30
*** maishsk_afk has quit IRC14:31
jeblairAJaeger, fungi: at any rate, i'm definitely working on clarifying the situation; i want to see where i can get agreement with the tc on it.  that will probably take a couple of weeks.  until then, i would say that technically nothing has changed, but it would be good to ask projects to start in openstack if they can, and if there is some reason they must be in stackforge, let them14:31
*** nihilifer has joined #openstack-infra14:32
jeblairAJaeger, fungi: doing so thus far has given us a lot of useful information (and i think worked out well for some projects)14:32
fungijeblair: yep, we've not been insisting, just explaining how easy it is to "just start in openstack"14:34
*** dboik_ has quit IRC14:34
fungijeblair: also https://review.openstack.org/19355014:34
*** jasondotstar has quit IRC14:35
*** cgoncalves has left #openstack-infra14:35
*** achanda has quit IRC14:36
openstackgerritMerged openstack-infra/project-config: zuul: actually enable puppet4 voting  https://review.openstack.org/19337914:36
*** maishsk_afk has joined #openstack-infra14:36
*** maishsk_afk has quit IRC14:37
*** jasondotstar has joined #openstack-infra14:37
*** rlandy has quit IRC14:39
*** sarob has joined #openstack-infra14:39
openstackgerritMonty Taylor proposed openstack-infra/shade: Add IPv6 to the server information too  https://review.openstack.org/19358114:40
mordredjeblair, fungi, Shrews, dguerri: ^^ while poking at that section, figured I should add ipv6 info into the server record while we were at it14:41
*** tvanderwiel has quit IRC14:41
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: add query for keystone wsgi restart issue  https://review.openstack.org/19357114:41
hasharmordred: hello :) Will you ever upstream your diskimage-builder puppet element ? :D14:41
mordredhashar: hrm. that's a great idea.14:42
mordredhashar: so - let's say "yes" - and if the dib-nodepool stuff goes well today, I'll see if I can work on that next14:42
hasharmordred: I guess once it is stable enough, you probably want to move it from puppet to "upstream"14:42
mordredhashar: I don't know if you saw - but our ansible module for running puppet is now in upstream ansible14:42
hashar\O/14:42
hasharcongratulations!14:42
mordredyah I'm exciteda bout that14:43
hasharwe have discussions regarding salt/puppet/ansible etc on our side. But there is no firm consensus yes14:43
hasharyes/yet14:43
hasharmeanwhile, I got nodepool setup on our system.  Tweaking the disk image to have something eventually working for us :}14:43
*** _nadya_ has quit IRC14:43
*** miqui has joined #openstack-infra14:43
*** _nadya_ has joined #openstack-infra14:44
*** sarob has quit IRC14:44
*** dguerri is now known as dguerri`14:44
mordredhashar: well, I'm pretty pleased with ansible+puppet so far ... and we'll hopefully soon have moved from ansible+puppet-agent+puppetmaster to ansible+puppet-apply - which I think will be very nice14:44
*** krtaylor has joined #openstack-infra14:45
mordredhashar: reduces the bootstrapping - with either salt or puppet agent, you need an additional agent on the machine that you need to enroll in the certificate management14:45
*** yamahata has joined #openstack-infra14:45
mordredhashar: salt + puppet agent was the worst - two different certs and two different agents :)14:45
hasharso you just crontab the ansible/puppet-apply run ?14:46
mordredyup14:46
hashardoes ansible has anything to run stuff to all machines? (we use salt)14:46
mordredour cron job runs ansible, which runs puppet in sequence on the node where we need a certain order, and then in parallel on the others14:46
mordredit does14:46
mordredit's very good at running stuff on all machines14:46
*** kushal has joined #openstack-infra14:46
fungiit reminds me of the "parallel ssh" tools which we used to use on high-performance computing clusters to run specific commands on every cluster member from the cluster master14:47
mordredhashar: otoh - if you're already using salt for that and you're happy with it, salt would also do a fine job - but we're going to have a bit soon with ansible to deal with copying a node's hieradata around appropriately14:47
mordredfungi: it really does14:47
*** yamamoto_ has quit IRC14:48
*** _nadya_ has quit IRC14:48
hasharmordred: I am far from those pre occupations to be honest. I use whatever is provided by our sysadmins :D14:48
*** bdemers has joined #openstack-infra14:48
fungii like that ansible reduces well to the simple case of "here's a list of machines to which i have assh access and a command i want you to run now on all of them... let me know how it turns out"14:48
mordredhashar: this is a good choice14:49
*** dvorak is now known as clayton14:49
*** bhunter71 has joined #openstack-infra14:49
*** pblaho has joined #openstack-infra14:50
*** dguerri` is now known as dguerri14:50
*** yamamoto has joined #openstack-infra14:53
*** krtaylor has quit IRC14:53
*** Longgeek has quit IRC14:53
*** Longgeek has joined #openstack-infra14:54
*** mmedvede has quit IRC14:54
*** sabeen1 has joined #openstack-infra14:55
*** clu_ has joined #openstack-infra14:56
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Pin to a specific fedora mirror  https://review.openstack.org/19356014:56
openstackgerritChristopher Aedo proposed openstack-infra/system-config: Add app-catalog-admin mailing list  https://review.openstack.org/19147814:57
*** sarob has joined #openstack-infra15:01
openstackgerritMerged openstack-infra/shade: Catch all exceptions around port for ip finding  https://review.openstack.org/19354015:02
*** ala_ has quit IRC15:03
*** _nadya_ has joined #openstack-infra15:05
*** k4n0 has quit IRC15:05
*** yamamoto has quit IRC15:06
*** mmedvede has joined #openstack-infra15:10
*** dboik has joined #openstack-infra15:10
*** esker has joined #openstack-infra15:11
mordredsdague: in a nova+neutron context with multiple network sources possible from neutron, do floating ip pools still make sense? like, are they a thing at all?15:12
*** dangers_away is now known as dangers15:12
mordredsdague: it seems like the two concepts fill a similar space, and I'm torn on what to do about it in the floating-ip abstraction code in shade15:12
*** armax has joined #openstack-infra15:13
*** yamamoto has joined #openstack-infra15:13
*** jsavak has joined #openstack-infra15:14
dguerrimordred: fwiw: when the cloud has neutron, neutron external networks _are_ floating ip pools.15:14
mordreddguerri: they are exposed as pools in the nova passthrough interface?15:14
dguerrimordred: afaik yes15:15
* dguerri is doublechecking15:15
* mordred goes to check vexxhost behavior15:15
*** mriedem has quit IRC15:16
*** mriedem has joined #openstack-infra15:16
mordreddguerri: yes. I verify this15:16
dguerriyes15:16
dguerriacc, too slow :)15:16
* mordred wonders why subnets aren't exposed as floating ip pools ... but decides to drop it15:17
dguerrimordred: good point15:17
*** tvanderwiel has joined #openstack-infra15:18
*** mtanino has joined #openstack-infra15:18
mordredok. I've also confirmed that there is a default floating-ip pool on auro, which uses nova-net with floating ips15:19
mordredAND that HP does not expose nova floating-ip-pool-list15:21
mordredat least not to me15:21
* mordred does a little dance15:21
dguerrimordred: that's also a big difference between nova-net and neutron: when you allocate a new floater, you can omit the pool for nova (getting it from the default pool) but you can't omit the net_id for neutron15:21
*** nihilifer has quit IRC15:22
mordredwow. that's amazing15:22
mordredhave I mentioned I dislike floating ips in general?15:22
dguerrimordred: I addressed that here: https://review.openstack.org/#/c/183555/5/shade/__init__.py15:22
dguerri#1824515:23
dguerrisorry #182415:23
*** yamamoto has quit IRC15:23
mordreddguerri: yes. awesome15:24
mordredso - I think the logic and functionality is all spot on15:24
mordredI think the main question is - do we use the words "network_or_pool" - or do we just say "network" and have folks using teh pool interface hand their pool name as a param15:25
mordred(I ask because we normalize nova networking concepts to neutron shape frequently)15:25
*** cdent_ has joined #openstack-infra15:25
*** koolhead17 has quit IRC15:25
*** koolhead17 has joined #openstack-infra15:26
*** jasondotstar has quit IRC15:26
dguerrimordred: I am fine either way. But I feel a subtle pleasure thinking we could just pretend nova-network doesn't exists :-)15:27
*** jasondotstar has joined #openstack-infra15:28
openstackgerritAntoine Musso proposed openstack-infra/nodepool: Fix enable_node and disable_node methods  https://review.openstack.org/19360015:28
*** otter768 has joined #openstack-infra15:28
Shrewsmordred: dguerri: maybe just network, and explain in the docstring it can be pool for nova15:28
*** romcheg has quit IRC15:28
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Allow the use of git add !!null as an `scm` value  https://review.openstack.org/19360115:29
*** cdent has quit IRC15:29
*** cdent_ is now known as cdent15:29
dguerriShrews: ++15:29
mordredShrews: ++15:29
mordredsold15:29
Shrewsmordred: dguerri: and for instance vs. port?15:30
dguerrilet's get rid of it15:30
dguerrimordred: Shrews if needed we can add something later15:31
dguerrimordred: Shrews removing stuff, is trickier, tho15:31
mordred++15:31
mordreddguerri: I don't think it's currently used, right?15:31
dguerriit's not15:31
Shrewsdguerri: i'm good with that15:32
mordreddguerri: well, that said ... in nova-net, instance is the server id, right?15:32
dguerrisold15:32
dguerriyes15:32
mordreddguerri: in neutron, we get the port id and should be able to infer the server id from that, no?15:32
dguerriyes15:32
dguerriah15:32
dguerriI see15:32
mordredI ask, because having the server_id (possibly always called "server_id") is useful15:32
dguerrido you wan't the server id15:32
mordredI've never once thought about ports when wrking with floating ips15:33
dguerriit need an API call, tho15:33
mordredit does?15:33
dguerri*needs15:33
mordredit's not in the port struct?15:33
*** otter768 has quit IRC15:33
*** mfisch has joined #openstack-infra15:33
mordredoh - because the floating struct has port_id, so we'd need to look up what server that's on15:33
dguerriyes, unfortunately...15:34
mordreddguerri: if it's not attached, it will not have a port id, right?15:34
dguerricorrect, it is None15:34
*** mfisch has left #openstack-infra15:34
mordredhrm15:34
dguerrihold on15:34
dguerrinot always15:34
mordredso - "is_attached" is an important piece of information ... oh awesome15:34
mordredlet's do what you said and omit it for now15:34
dguerriagreed15:34
mordredand then go to the nodepool code and see what our consumption there needs to be successful - like, if we were to use shade floating ips in nodepool15:35
dguerria port could actually live without a server15:35
mordredand that should give us a good sense of where/if we need to make an extra call to fill this out, or if it's fine15:35
*** yamamoto has joined #openstack-infra15:35
mordredbut I think we can land this without for now15:35
*** MarkAtwood has joined #openstack-infra15:35
dguerriok, patching the patch now15:36
Shrewsyay! my question brought about changes and concensus. thx dguerri, mordred15:36
dguerriShrews: you are very welcome15:36
*** jsavak has quit IRC15:37
ttxmordred: got 5 minutes to talk Maniphest ?15:37
*** EricGonczer_ has joined #openstack-infra15:37
dguerriI would actually ask those questions before sending out the review, but we live in different TZs :D15:37
mordredttx: yes!15:37
*** jsavak has joined #openstack-infra15:38
*** abregman has quit IRC15:38
ttxmordred: so... looking into that CustomField thingy15:38
ttxhttps://secure.phabricator.com/book/phabdev/class/ManiphestCustomField/15:39
uberjayasselin: well, finally ran into that floating ip leak issue, and apparently hadn't put the relclaim cron job in. d'oh....15:39
mordredttx: for security things yeah?15:39
ttxNo, for tracking repo/branch15:39
mordredah15:39
mordredyes15:39
ttx(so that we can use "project" as free form tags15:39
mordred(sorry, context switching things in)15:39
ttxand build random dashboards freely as-needed)15:40
mordredyup - which would be neat15:40
mordredpeople like random dashboards15:40
ttxOn paper, it seems CustomField is plenty powerful enough15:40
ttxI just have a hard time picturing how to use it15:40
mordreddo you have to add custom fields via php?15:41
ttxyou apparently shoudl drop a class in some random folder and voila15:41
openstackgerritAntoine Musso proposed openstack-infra/nodepool: Fix enable_node and disable_node methods  https://review.openstack.org/19360015:42
ttxbut reading PHP makes my head hurt. I suspect I could work it out from a working install though15:42
*** EricGonczer_ has quit IRC15:42
ttxmordred: was wondering what the best next step would be for us there15:42
mordredttx: it seems there is a "maniphest.custom-field-definitions" config setting15:42
mordredttx: http://15.126.194.141/config/group/maniphest/15:42
*** dizquierdo has quit IRC15:42
ttxyes but that is pretty basic from what I read15:42
mordredah15:43
ttxin particular doesn't use shouldAppearInListView() so is invisible in lists15:43
ttxwhere I think that would be great to have them to emulate story tasks15:43
*** flepied1 has quit IRC15:43
ttxbut I'm at level 0.1 of exploring so I won't pretend I have all the keys here15:44
pelixlooking for another review for JJB to see if the following changes can be approved: https://review.openstack.org/183488, https://review.openstack.org/133997, https://review.openstack.org/181541, https://review.openstack.org/18015015:44
ttxmordred: see https://secure.phabricator.com/book/phabricator/article/custom_fields/15:44
pelixzaro: you around to review https://review.openstack.org/#/c/180150/? other three above need someone else besides you or me to approve15:45
ttxyou can configure easy ones in JSON. FOr complex ones you need to write a class15:45
ttxmordred: but then the class sounds not that complicated, since by default it behaves like a basic field and you just override extra methods15:45
ttxsee bottom of that last page15:46
*** laughterwym has quit IRC15:46
ttxWas just wondering what's the best way to play with it. How easy is it to fire your own ?15:46
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: Add several jobs for Manila periodic pipeline  https://review.openstack.org/19353815:46
mordredttx: (reading)15:47
mordredttx: it's very easy to fire your own ... it's slighly harder to fire your own _and_ import useful dataset - but I can get you a copy of one pretty easily15:48
*** krtaylor has joined #openstack-infra15:48
ttxmordred: bah, I don't need dataset15:49
ttxmordred: I guess it's simpler I just set up one and try that advances Custom field thing, unless you beat me to it15:49
*** leopoldj has quit IRC15:49
*** mmedvede has quit IRC15:50
*** marzif has joined #openstack-infra15:50
mordredttx: I usd this: https://github.com/openstack-infra/puppet-phabricator to set up the one I have running right now15:51
mordredttx: there is a readme in there with some more notes15:51
*** marzif has quit IRC15:51
ttxhmm, ok, hopefully will have some time soon. I would really like to explore how we can solve that before we go any further15:52
*** marzif has joined #openstack-infra15:52
ttxit's a pretty basic field(s) but ideally would need to appear prominently to be useful15:53
ttxand I have no idea how powerful that public function renderOnListItem($view) is15:54
ttxdocumentation is pretty minimal15:54
mordredyah - I think it'll take poking for sure15:55
mordredttx: my next thing after I get nodepool-dib sorted is to jump on this15:55
ttxok, feel free to beat me to it15:55
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Add Parameterized Remote Trigger Plugin support.  https://review.openstack.org/15032715:56
*** bexelbie has quit IRC15:57
Shrewsdguerri: i was looking at one of mordred's shade changes and noticed he is used the object form to address server info. A lot of the new IP code is doing similar things for servers (and we were bad reviewers and didn't catch it). We need to start using the dict form so we can phase out the object form.15:57
mordredShrews: nod.15:58
dguerriShrews: yep... how much have you found of that?15:58
*** marzif has quit IRC15:58
dguerricould someone explain the meaning of floating IP status in neutron? it seems to be used differently in different openstack version :-S but it could be just me15:59
Shrewsdguerri: several places. some old, some new15:59
Shrewsdoh15:59
mordredmarkmcclain: ^^ any chance you can tell us where to look to learn about that?15:59
Shrewsoh, yeah. right channel15:59
*** lascii is now known as alaski15:59
dguerri:D15:59
jeblairfungi, mordred: i tested https://review.openstack.org/193368 with zuul tests, and they did not fail16:00
jeblairfungi, mordred: i think we can merge/release;  then a zuul restart should pick it up and hopefully we won't see the demand gap16:00
Shrewsdguerri: we need to put that on our todo list to change those16:00
dguerriShrews: ++16:00
*** RichardRaseley has joined #openstack-infra16:01
*** bexelbie_ has quit IRC16:02
*** jlanoux_ has quit IRC16:02
fungijeblair: cool, i'm +2 on it in that case16:02
mordredjeblair: ++16:02
*** RichardRaseley has quit IRC16:03
*** RichardRaseley has joined #openstack-infra16:03
*** RichardRaseley has quit IRC16:03
*** yamahata has quit IRC16:03
fungibased on the cacti graph, we should in moments be entering the overflow condition again16:04
*** RichardRaseley has joined #openstack-infra16:04
jeblairoh drat; shoulda done this 30 mins ago16:05
*** esker has quit IRC16:05
*** markus_z has quit IRC16:06
*** mmedvede has joined #openstack-infra16:06
openstackgerritMerged openstack-infra/gear: Do not raise RetryIOError on blocked write  https://review.openstack.org/19336816:06
*** e0ne has quit IRC16:10
*** dannywilson has joined #openstack-infra16:11
jeblairtagged 0.5.816:11
*** mpaolino has quit IRC16:11
*** pbourke has quit IRC16:13
pleia2good morning16:13
*** pbourke has joined #openstack-infra16:13
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Allow the use of git add !!null as an `scm` value  https://review.openstack.org/19360116:14
*** ujuc has joined #openstack-infra16:14
*** gyee has joined #openstack-infra16:14
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Enable parameter ordering in trigger_parameterized_builds  https://review.openstack.org/17504916:15
anteayamorning pleia216:15
*** ashleighfarnham has joined #openstack-infra16:16
*** derekh has quit IRC16:17
jeblairfungi: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-12-09.log.html16:18
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Update 'timeout' wrapper module  https://review.openstack.org/12946716:18
jeblairfungi: that was our discussion where we decided to use two servers instead of growing the haproxy server16:18
fungijeblair: oh! good find16:18
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Allow the use of !!null as an `scm` value  https://review.openstack.org/19360116:19
jeblairthat seems to suggest that we believe backend distribution is based on ip addresses, not backend load16:19
*** koolhead17 has quit IRC16:19
fungiyeah, i don't think that haproxy is currently configured to be at all smart about it and is simply hashing the requester's ip address and destination port16:20
fungiand using that to choose a backend16:21
fungiat least that's my recollection but i need to go back through the config and manpage16:21
jeblairyep, "balance source"16:21
*** hashar has quit IRC16:21
jeblairis what we have configured16:21
*** Guest70353 has quit IRC16:22
*** arxcruz has quit IRC16:22
*** mriedem is now known as mriedem_burgers16:22
*** achanda has joined #openstack-infra16:23
*** ildikov_ has quit IRC16:23
openstackgerritDavide Guerri proposed openstack-infra/shade: Add Neutron/Nova Floating IP list/search/get  https://review.openstack.org/18349616:24
*** RichardRaseley has quit IRC16:24
clarkbI am pretty sure git is good about one connwction per command now and we can change that if desireable16:25
clarkbeven when using non smart gttp16:25
dguerriShrews: mordred : I had to add an additional attribute 'attached'. It turned out I was using port_or_instance_id to determine if a floater was attached... so ...16:26
*** RichardRaseley has joined #openstack-infra16:26
dguerriShrews: mordred : ^ review is the above16:26
*** romcheg has joined #openstack-infra16:27
*** sdake_ has joined #openstack-infra16:27
jeblairclarkb, fungi: so here's my current (incomplete) thinking: we lower maxconn for the backends to prevent them from being overloaded.  if we stick with 2 fe's, then we probably need to be conservative with that since each fe only knows about potentially half the connections (though, each fe is "git.o.o" for 5 mins at a time, so maybe it's not that far off)16:27
jeblairclarkb, fungi: we might consider switching to least-connections, and moving to a single fe, which would then make the maxconn accounting more accurate16:28
jeblairclarkb: and it sounds like you were saying you think that might be reasonable to do?16:28
*** dguerri is now known as dguerri`16:29
*** igorbelikov has quit IRC16:29
clarkbjeblair yup as git clients are well behaved and dont seem to make multiple connections per fetch/clone16:29
*** vponomaryov has quit IRC16:30
mordreddguerri`: looks great!16:30
clarkbthe source balance method was being conservative so that multiple connections per client would be safe16:30
*** rossella_s has quit IRC16:30
*** rossella_s has joined #openstack-infra16:30
*** sdake has quit IRC16:31
*** dannywilson has quit IRC16:31
*** RichardRaseley has quit IRC16:31
*** RichardRaseley has joined #openstack-infra16:31
*** devvesa has quit IRC16:31
*** RichardRaseley has quit IRC16:32
*** dannywilson has joined #openstack-infra16:32
*** fedexo has joined #openstack-infra16:32
*** jsavak has quit IRC16:33
jeblairclarkb, fungi: okay, options: a) leave everything as is, except set maxconn on backends (and probably spin up more backends);  b) switch to single haproxy, maxconn on backends, least-connections lb algorithm16:33
jeblairanything else?  which one do you like?16:33
anteayajeblair: would option b) include more backends as well?16:35
*** olaph has joined #openstack-infra16:35
*** yamamoto has quit IRC16:36
jeblairanteaya: not sure; it would spread load better which may make better use of the ones we have16:36
anteayavery good, thanks16:36
* anteaya listens16:36
*** dannywilson has quit IRC16:36
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Validate the use of `[]` as an `scm` value  https://review.openstack.org/19360116:36
jeblair(whereas with a, the inefficiency makes needing more backends more likely)16:36
anteayaah yes16:37
*** rm_work is now known as rm_work|away16:37
jeblairit looks like a 30G compute-optimized instance will get us 1GB of outbound bandwidth, which is slightly less than what we have now (1.2), but more than we need (mean 0.16, median somewhere around 0.2, peak 0.6-0.7)16:38
*** asahlin is now known as asahlin_away16:38
anteayado we have a quota on how many of those instances we can use?16:38
jeblairyes, but if we ask nice, they might raise it16:39
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Validate the use of `[]` as an `scm` value  https://review.openstack.org/19360116:39
anteayaokay great16:39
clarkbjeblair we can easily try both and use dns to move between them16:39
jeblairclarkb: git-fe03? :)16:39
anteayaclarkb: spin up an entire second git farm?16:39
jeblairanteaya: we only need the fe, not backends16:39
clarkbjeblair ya something like that16:39
anteayajeblair: ah okay16:40
fungiyeah, i'd just add a third one and then spin down the other two, doing it all through dns updates16:40
clarkbor just git-fe.o.o16:40
anteayaguess I thought front ends were tied to specific backends16:40
clarkbnot really since they are read only we can share backends16:42
anteayawonderful16:43
jeblairokay, i'll see if i can come up with a good maxconn value for backends and set that everywhere, then spin up a 30g compute instance and configure least-conn on it only16:44
sc68calmordred: hey I read scrollback about the floating IP stuff - let me know if there is anything else I can help answer16:44
sc68calmordred: I too, dislike floating IPs - all hail IPv6!16:44
*** dguerri` is now known as dguerri16:45
mordredsc68cal: did we say anything that sounded completely terrible?16:45
mordredsc68cal: ++16:45
mordredzomg. ipv6 == so much better16:45
*** ryanpetrello has quit IRC16:45
sc68calmordred: no it sounds like we found another nova-net and neutron mismatch in the API :(16:45
fungisc68cal: i only need these dirty old ipv4 addresses for a few more years, i hope ;)16:45
sc68calfungi: haha16:46
*** dguerri is now known as dguerri`16:46
mordredsc68cal: well, to be more precise, we found another place where neutron has a nicer api16:46
*** ryanpetrello has joined #openstack-infra16:46
sc68calmordred: heresey!16:46
* sc68cal falls on his fainting couch16:47
*** jcoufal has quit IRC16:47
* mordred hands sc68cal a mint julep and a fan16:47
fungii'm disappointed that here it is 2015 and my residential broadband isp provides ipv6 by letting me configure a "6rd" tunnel to a broker on their network border16:47
mordredfungi: I'm disappointed by many things16:48
*** ZZelle_ has joined #openstack-infra16:48
sc68calfungi: ouch.16:48
fungii want a full mtu for my v6 traffic, darn it!16:48
*** rlandy has joined #openstack-infra16:48
*** jsavak has joined #openstack-infra16:48
fungigimme slaac/dhcp6/rtsol/bgp6/something native16:49
yolandajeblair, thx for the quick fix and release of gear16:49
jeblairyolanda: np; i'll probably restart our zuul later today to pick it up16:49
yolandawe are heading for that tomorrow16:49
*** puranamr has joined #openstack-infra16:49
fungijeblair: in good news, we may have avoided the gearman outage today because of new shade. or at least something seems to have caused us to _still_ not started uploading our dib results16:50
yolandajeblair, did you ever hit that bug, or that's an special case we were facing?16:50
*** ujuc has quit IRC16:50
fungimordred: have you checked up on image updates?16:50
*** _nadya_ has quit IRC16:50
*** puranamr has quit IRC16:50
*** puranamr has joined #openstack-infra16:51
*** _nadya_ has joined #openstack-infra16:51
jeblairyolanda: we're pretty sure we see it when we upload dib images, because they use so much bandwidth16:51
eantyshevHello! Does anybody here run many CI accounts? AFAIU, they should be run in separate Zuul nodes. Do you have many Jenkins nodes too?16:51
*** ZZelle__ has joined #openstack-infra16:51
fungias soon as i've finished my lunch i'll see if i can tell if it's still going or broke16:51
*** vnod has joined #openstack-infra16:51
asselinuberjay, fyi this merged https://review.openstack.org/#/c/191216/ and this needs review...might interest you: https://review.openstack.org/#/c/192819/16:52
*** sdake has joined #openstack-infra16:52
asselineantyshev, I have a few accounts. they all run independently16:52
yolandajeblair good to know, and thx for that help yesterday16:52
mordredfungi: I was also going to check it after lunch16:52
uberjayasselin: ah, that's a great idea. thanks for the pointer.16:52
jeblairnp, thank you :)16:53
*** yamahata has joined #openstack-infra16:53
mgagneI'm looking at nodepool and I'm wondering at what time is the keypair injected into the image. Is it done by dib or nova boot? if nova boot, does nodepool assume a keypair is pre-created and available via Nova api?16:53
asselineantyshev, I tried sharing, but it was too complicated.16:53
*** rm_work|away is now known as rm_work16:53
eantyshevasselin: So you have independent Jenkins nodes?16:53
uberjayasselin: i'm a bit behind, as the floating ip pool was full since earlier this morning... the queue is ~10 jobs behind16:53
greghaynesdguerri`: https://review.openstack.org/#/c/183496/ failure is wierd16:54
*** thedodd has joined #openstack-infra16:54
asselinuberjay, right this patch can help, in case any of them merged already, but still queued: https://review.openstack.org/#/c/192819/16:54
greghaynesdguerri`: that got modified in the parent patch, not sure why it'd fail there but not in parent16:54
greghaynesdguerri`: er, sorry, that test fail is wierd, not the patch ;)16:54
asselineantyshev, yes, each their own jenkins master and nodepool slaves16:54
mordredmgagne: for dib images, it's done by dib16:54
mgagnemordred: is it done by a custom element or is an env var set?16:55
eantyshevasselin: thank you for that information!16:55
mordredmgagne: our puppet, which we run in our dib, adds it16:55
*** _nadya_ has quit IRC16:55
*** sdake_ has quit IRC16:55
mgagneoh, thanks for the info!16:55
mordredmgagne: although there is a dib element (local-config I think?) which will inject a local ssh key if you want for doing testing of things16:56
greghaynesWell, itll need to get injected with the correct user and such, I tend to use devuser for that16:57
greghaynesthe devuser element, that is16:57
mordreddevuser16:57
mordredthat's it16:57
*** Swami has joined #openstack-infra16:57
waynrclarkb: would you mind looking at https://review.openstack.org/#/c/183488 ?16:58
*** bradjones has quit IRC16:58
mgagneso supported methods are: dib element (devuser), puppet, node keypair (config nodepool)16:58
waynrno rush I just noticed your last comment indicating that you would eventually review and approve16:58
*** bradjones has joined #openstack-infra16:58
*** bradjones has quit IRC16:58
*** bradjones has joined #openstack-infra16:58
*** yamahata has quit IRC16:59
*** mwagner_afk has quit IRC16:59
*** ddieterly has quit IRC17:00
*** doug-fish has joined #openstack-infra17:01
mordredmgagne: well, node keypar really only works if you're using not-dib and template nodes17:01
mgagnereally? what if I install cloud-init?17:02
mordredmgagne: it's conceivable we could add keypair support to dib-based image boots - it would not be hard to do - but we do not currently use that17:02
mordredmgagne: nodepool doesn't provide any keypair information to nodes it boots17:02
mgagnewth17:02
mordredbecause it's never been an interesting thing to us17:02
mgagnehttp://docs.openstack.org/infra/nodepool/configuration.html keypair17:02
mordredwe have already pre-created the images in both sides17:02
mordredso injecting them at boot time is not really a benefit17:03
*** RichardRaseley has joined #openstack-infra17:03
openstackgerritVinod Pandarinathan proposed openstack-infra/system-config: Add openstack cloudpulse channel bot  https://review.openstack.org/19364317:03
fungimgagne: that tells it what keypair to use when connecting to a template instance, when customizing it in preparation to make a snapshot (used for old-style non-dib nodes)17:03
mgagnenow that's a confusing config name17:03
fungimgagne: basically a bootstrapping key17:03
mordredagree17:03
*** ashleighfarnham has quit IRC17:03
mordredit should also get some better docs to indicate what it's there for17:04
devanandahey, so apparently I missed a conversation in the early hours of this morning re: whether young/new/whatever projects should go on stackforge/ or openstack/ namespace17:05
*** sdake_ has joined #openstack-infra17:05
mordreddevananda: tl;dr - there is no reason for them to go into stackforge/17:05
mordreddevananda: there is no new policy on the subject, that is under discussion17:05
greghaynes\o/ less renaming17:06
mordreddevananda: but it is not required, nor is there benefit, for them to go to stackforge17:06
devanandamordred: sure. except if they're not part of the existing team, then they dont belong in openstack/ under the umbrella of said team17:06
mordreddevananda: there is nothing stopping them from being a new team17:06
jeblairs/team/project/ but yeah :)17:06
devanandamordred: do we have a policy for leaderless teams in openstack/ ? or just forming a new team?17:06
devanandajeblair: no, I mean team17:06
mordredjust form a new team17:06
openstackgerritMerged openstack-infra/shade: Centralize exception management for Neutron  https://review.openstack.org/19218117:06
mordredyay ^^ I like that patch17:07
*** tvanderwiel has quit IRC17:07
greghaynesmordred: did you see my comment on the parent patch? it has me a little worried17:07
mordredgreghaynes: I did not17:07
mordredbut I'm going to AFK for lunch17:07
mordredso I'll look in a minute17:07
greghaynesmordred: basically - the parent patch failed a test that looks like it was a bug in the dependent patch17:07
greghaynesok17:07
*** sdake has quit IRC17:09
*** ddieterly has joined #openstack-infra17:09
fungidevananda: ideally the policy for just forming a new project team is simple enough that people don't need to spend much time debating whether it's worth doing. i note that the searchlight proposal made it from motion uploaded to gerrit to approved by the tc in a mere 6 days17:09
asselinfungi, is today a better day to merge this? https://review.openstack.org/#/c/183422/17:09
jeblairdevananda: i mean simply to point out that to add a new "team" you propose a "project" :)17:10
asselinfungi, and this? https://review.openstack.org/#/c/183429/17:10
jeblairi suspect for the purposes of this conversation, they are largely synonymous though.17:10
devanandajeblair: https://review.openstack.org/#/c/191142/2/reference/projects.yaml isn't proposing a new team17:10
devanandajeblair: and its first iteration did not either -- it proposed (and still does) adding a new project to an existing team17:10
jeblairdevananda: yeah, that's proposing a new repo for an existing project/team17:10
*** jtomasek has joined #openstack-infra17:11
*** bradjones has quit IRC17:11
jeblairdevananda: if you don't want it, don't accept it, and they can propose a new project/team :)17:11
fungiasselin: looking17:11
devanandajeblair: exactly what we did. now for questoin #2 -- what if we don't like the direction it's going?17:12
*** ashleighfarnham has joined #openstack-infra17:12
devanandaI believe the answer is: that's OK. it's a big tent.17:12
fungijeblair: devananda: also if you accept it now, doesn't mean it can't spin out to be under a separate project team later, and if it's created with a new project team now that doesn't mean that it can't be absorbed into another one later. those are just governance changes at that point17:12
devanandafungi: *nod*17:12
*** bradjones has joined #openstack-infra17:12
*** bradjones has quit IRC17:12
*** bradjones has joined #openstack-infra17:12
jeblairdevananda: yeah, if they aren't part of your project/team, er, ask nicely?  post to the mailing list?  if you think it's a really bad idea to start with, ask that the project be rejected at the start because they aren't collaborating or something?17:13
jeblairdevananda: but yeah, general answer is probably as you say.17:14
devanandabig tent allows project (teams) to exist and compete. but we haven't had a case before, afaik, where within a project, a subcomponent (like the UI) had competing implementations17:14
jeblairi believe the tc is willing to exercise some discretion here17:14
devanandajeblair: sounds like we're on the same page :)17:14
fungidevananda: yeah, that would be as crazy as a project subcomponent like networking having competing implementations ;)17:15
*** claudiub has quit IRC17:15
jeblairdevananda: so if a project competes with something in a way that you think is harmful to the project, i think the tc would be willing to talk about it.17:15
devanandafungi: totally crazy :)17:15
fungithat way lies madness17:15
jeblairdevananda: is ironic-dashboard competing with something?17:15
*** nihilifer has joined #openstack-infra17:16
*** abitha has joined #openstack-infra17:16
jeblairdevananda: (from my pov, i hear things like "ironic wants to run standalone" so an "ironic-dashboard" project shows up and i think "oh it must be part of that").  if i need re-education, i'm ready to recieve it.  :)17:17
devanandajeblair: yup. see ML thread about all this, to which I have not replied yet, but clearly need to17:17
devanandajeblair: tldr; I wasn't aware of this new project before it was proposed to openstack/governance17:18
anteayadevananda: wow17:18
*** puranamr has quit IRC17:18
fungithat's indeed a bit jarring17:18
devanandayup17:18
anteayadevananda: yes when I saw the patch fly past with ironic in the title I figured you were aware if not behind it17:18
devanandaanteaya: I was not.17:18
anteayadevananda: thanks for telling us17:19
devanandathe big tent allows for anyone to propose changes to governance. so I'm happy that folks are taking up their rights. but this wasn't socialized properly17:19
devanandanone of the current ironic TEAM are behind or involved in this17:19
devanandaand that is my only problem with it17:19
jeblairdevananda: is there another ironic-dashboard project?  or is there a panel in horizon?17:20
fungiwell, if the projects it fronts and integrates into aren't aware of its existence, that suggests that at a minimum open design is missing from their team methodology17:20
jeblairdevananda: where is the ironic team focusing ui work?17:20
jeblairfungi: indeed17:20
devanandajeblair: there was some (now abandoned) work from a few teams last cycle. I've been working with krotscheck (he's really been doing all the work) on new things for the past few months17:21
fungithis is one of the places i fear we've still somehow sent the wrong message with our governance restructuring, that teams think they need to have something designed and working _before_ they bring it to the community for consideration17:21
*** mriedem_burgers is now known as mriedem17:22
*** rguillebert has quit IRC17:22
devanandajeblair: we talked about it on friday in YVR. folks in our community know about it. there's nothing in gerrit yet, though there is a working POC on github17:22
jeblairdevananda, krotscheck; my understanding from krotscheck's ml post is that krotscheck is working on plumbing/design, right?17:22
jeblairdevananda: there's a what on where?17:23
devanandaproof of concept17:23
*** jsavak has quit IRC17:23
jeblairdevananda: can you provide a link to that?17:23
fungiplease explain this "gi-thub" of which you speak17:23
devanandaheh17:23
*** jsavak has joined #openstack-infra17:23
devanandajeblair: yes, but not sure how it's relevant to the discussion17:24
* fungi read "there's some random code sprayed somewhere out on the internet"17:24
*** zul has quit IRC17:25
devanandafungi: so this proposal from niuzhenguo was a 2,500 line code drop on github with one contributor17:25
*** zul has joined #openstack-infra17:25
fungiyeah, that's pretty crazy17:25
JayFdevananda: where's the github repository that you'er all speaking about?17:26
devanandai assume you looked at the upstream: ref17:26
devanandahttps://github.com/niuzhenguo/ironic-dashboard17:26
*** notnownikki has quit IRC17:26
fungiclearly this is something written somewhere at a service provider/integrator now being "opened"17:26
devanandaclearly17:26
devanandawhich is great. I'm delighted they wanted to share17:26
openstackgerritMerged openstack-infra/jenkins-job-builder: Move writexml fix for older minidom to same module as caller  https://review.openstack.org/18015017:26
*** mrunge has quit IRC17:27
jeblairdevananda: okay, so here's my perception: huawei went and wrote a dashboard.  hp went and wrote a dashboard.  they are both on github.  public discussions about them are only now starting.17:27
devanandabut we need to colletively rewind and actually have a discussion within the project about whether we want to do this17:27
devanandajeblair: exactly17:27
devanandaand the discussion recently got sideswiped by 'should it be on stackforge or openstack'17:27
*** puranamr has joined #openstack-infra17:28
devanandawhich really isn't hte point :)17:28
jeblairdevananda: i think there is an opportunity for both teams to get together, and figure out a way to collaborate (or else, decide that they want completely different things and these are actually two projects)17:28
fungithis is from huawei? i'm having _no_ luck so far finding copyright from anybody17:28
jeblairdevananda: agreed; the stackforge/openstack thing is an inflection point where we discover things like this :)17:28
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: add query for keystone wsgi restart issue  https://review.openstack.org/19357117:28
jeblairfungi: er, well, someone advocating for it on the list has a huawei address17:29
*** otter768 has joined #openstack-infra17:29
devanandathe person who proposed the gov changes has an @huawei address17:29
fungijeblair: got it17:30
fungii mean, whoever put this thing on github seems to fairly consistently mark it as apache 2 license at least. just no real indication of provenance17:31
jeblairat any rate, i think it would good for _any_ project to start its life in a public manner, and using our project infrastructure.  it doesn't help anyone to go off and work on something in private and then be surprised that someone else is doing the same thing.17:32
*** ZZelle_ has quit IRC17:32
fungii do find a few files which were lifted from nova, so they still have "Copyright 2012 United States Government as represented by the Administrator of the National Aeronautics and Space Administration. All Rights Reserved."17:33
*** ZZelle__ has quit IRC17:33
*** otter768 has quit IRC17:33
openstackgerritMerged openstack-infra/elastic-recheck: add query for keystone wsgi restart issue  https://review.openstack.org/19357117:34
*** tqtran has joined #openstack-infra17:35
*** SumitNaiksatam has quit IRC17:36
*** yamamoto has joined #openstack-infra17:37
*** wenlock has joined #openstack-infra17:37
fungiasselin: i'm cool with the zuul log management puppet changes, but feel like we should wait to merge them until after jeblair restarts zuul later today, just so that if we do discover a problem with this it won't complicate/extend that restart17:38
*** e0ne has joined #openstack-infra17:38
asselinfungi, ok17:38
asselinfungi, I won't be around this afternoon, but I shouldn't be needed17:38
fungiasselin: yep. absolute worst case we revert it, but this is simple enough that we should be able to just iron out the bugs if there are any without reverting17:39
asselinfungi, agreed17:39
*** ghostpl has quit IRC17:40
devanandajeblair: I agree. though in practice, I find it easier to do a proof of concept on github, where I can delete the project if it turns out not to be a good idea.17:40
devanandajeblair: but pivot it to openstack/ as soon as the question "is this actually useful" is answered17:40
devanandaafaik, ya'll still can't delete projects, right?17:41
greghaynesmordred: so, a few things are wierd on this image17:41
greghaynesmordred: on the rax image with no networking, that is17:41
greghaynesmordred: first is the lack of logs from simple-init17:41
fungidevananda: we mark them as inactive/abandoned17:41
*** yamamoto has quit IRC17:42
fungitechnically we set them to "read-only" in gerrit17:42
jeblairthere's no good reason to delete code that's been written.  no one has to use it, and it may be useful to someone in the future.17:42
*** puranamr has quit IRC17:43
jeblair(even deleting something on github is illusory at best; the internet in general is really bad about deleting things)17:43
*** dtantsur is now known as dtantsur|afk17:43
devanandaindeed17:43
*** mpaolino has joined #openstack-infra17:44
fungi do sort of wish gerrit's api let you query/filter the repo list by config parameters17:45
fungier, i do17:45
jeblairdevananda: our community can handle experimentation.  if people wanting to start new efforts just said "hey, i want to try to do a foo" then we could get people coalescing around those ideas.  i'm not saying there is no place for private experimentation, but i think that the moment one thinks "i'll put this on github" is probably at or past the time one should actually be thinking "i will share this with my colleagues on the openstack proje17:45
*** Longgeek has quit IRC17:45
*** mpaolino has quit IRC17:45
anteayafungi: isn't that what hound is good for?17:45
*** cdent has quit IRC17:45
*** koolhead17 has joined #openstack-infra17:45
anteayafungi: though that doesn't help in finding a given patch17:46
anteayaas hound can't index proposed patches17:46
fungianteaya: in this case i'm talking about the gerrit configuration for each repo17:46
anteayaah17:46
openstackgerritMerged openstack-infra/tripleo-ci: Pin to a specific fedora mirror  https://review.openstack.org/19356017:46
anteayaso no way in there17:46
fungianteaya: the idea being "give me a list of active (not read-only or hidden) projects in gerrit"17:46
anteayaoh yeah17:46
*** mpaolino has joined #openstack-infra17:46
anteayagerrit only17:46
anteaya:(17:47
jeblairfungi: i bet that's not a super-difficult gerrit change17:47
fungianteaya: the project list api call doesn't filter on anything besides project name and pagination17:47
*** ildikov has joined #openstack-infra17:47
anteayayeah, less then what you want17:47
*** mpaolino has quit IRC17:47
fungijeblair: yeah, likely could be added as an option to the projects method17:48
*** ildikov has quit IRC17:48
*** ildikov has joined #openstack-infra17:48
*** tvanderwiel has joined #openstack-infra17:48
fungiinteresting that they _do_ have a rest api call to initiate git garbage collection per project though17:48
anteayainteresting17:48
*** mpaolino has joined #openstack-infra17:49
anteayaI didn't know git had garbage collection17:49
fungianteaya: man git-gc17:49
anteayathank yo9u17:49
devanandajeblair: what is the appropriate way to experiment on new projects in gerrit?17:49
fungidevananda: normally when someone on the infra team wants to do that, they just create a new git repo in gerrit within the infra team17:50
pelixfungi jeblair mordred: there are a few changes in JJB from both Khai or myself (or have been worked on by us) that need a +2 from another core reviewer assuming they look good.17:50
devanandafungi: with consummate proposals to projects.yaml, governance, etc ?17:50
*** mpaolino has quit IRC17:50
fungidevananda: yep. those are remarkably short and easy to churn out17:51
fungiespecially if you're starting with an empty repo and no-op jobs17:51
devanandafungi: maybe I'm missing something ... but it seems totaly unnecessary to involve the TC for every experimental project. and while on the TC, I dont recall ever seeing such things from infra17:51
*** puranamr has joined #openstack-infra17:51
devanandafungi: also - it's much easier for infra to do that, since infra can approve infra's changes to infra17:51
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Ensure tests run with ConfigParser object  https://review.openstack.org/17982217:51
devanandafungi: than, say, me waiting a few days for ya'll to approve a new project that i want to experiment on17:52
fungidevananda: this is one of the reasons we pushed to have governance changes adding repos to an existing project be administrivia and not need air time in tc meetings17:52
krotscheckSorry I'm late to the conversation, was relocating to Zoka17:52
fungiand also not block new project changes to project-config on them17:52
*** zxiiro has quit IRC17:53
fungidevananda: it's getting faster. we have something like 10 core reviewers on that repo now17:53
krotscheck(That's a coffee place)17:53
jeblairyep, at this point, a new repo for an existing team can be created as fast as the project-config folks review it.17:53
*** mpaolino has joined #openstack-infra17:53
*** marzif has joined #openstack-infra17:53
fungiand we can add more people to that fairly easily17:53
devanandajeblair: fungi: oh, that's great :)17:54
jeblairi can't remember the last time i +Ad my own repo creation change either; i usually wake up to find a repo there 2 or 3 days later17:54
jeblairsometimes 117:54
openstackgerritMerged openstack-infra/jenkins-job-builder: Change zuul-cloner parameters  https://review.openstack.org/17600817:54
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Ensure tests run with ConfigParser object  https://review.openstack.org/17982217:54
devanandakrotscheck: welcome back. I may step away soon, now17:54
krotscheckdevananda: Alrightey, what's left to discuss?17:55
yolandamordred, jeblair, do you have some feedback for the nodepool 2 queue change? about working on some approach to guarantee the rate17:55
*** jasondotstar has quit IRC17:56
yolandamordred, yesterday i had to go, my children were looking at me with angry eyes, so i lost your feedback if you gave me17:56
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Take parameter order from yaml in trigger_parameterized_builds  https://review.openstack.org/19366017:56
fungiyolanda: great news, we have logs! http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2015-06-18.log.html#t2015-06-18T17:29:3617:57
*** koolhead17 has quit IRC17:57
yolandaheh, fungi, good point17:58
fungiyolanda: though after a quick skim, i don't see them mention any suggestions after you left17:58
*** jasondotstar has joined #openstack-infra17:59
yolandayep, empty there17:59
*** jasondot_ has joined #openstack-infra17:59
jeblairyolanda: i don't have a suggestion.  but i will say that to get a +2 from me you will need to convince me that the change is necessary and worth the complexity.  i am not sure i believe it's needed at this point.18:00
*** mpaolino has quit IRC18:00
AJaegerdevananda: just reading the scrollback. Thanks for showing up and all the comments.18:00
yolandafungi, about the logrotate approach, i got -1 from Spencer and he is proposing a new approach of not using logrotate::fileremoval, but adding the absent functionality on the logrotate::file one18:00
anteayafungi: do we run git-gc on any gerrit repos?18:00
anteayafungi: I haven't seen it anywhere that I recall18:00
jeblairanteaya: we repack them18:00
anteayaah18:01
*** SumitNaiksatam has joined #openstack-infra18:01
jeblairanteaya: git repack -afd as i recall18:01
anteayathanks I'll re-read that section18:01
*** prad_ has joined #openstack-infra18:01
jeblairanteaya: you can probably find the configuration of it by searching for 'repack' in all of our repos because i have no idea where it is in puppet now.  (yay hound)18:01
anteayahehe18:02
yolandajeblair, using you simulator it was still reporting better times, even if hpcloud times were much better18:02
fungiyolanda: okay, interesting. mostly just wanting to make sure we can use puppet to clean up logrotate configs somehow when we remove/rename them in the manifest18:02
yolandai'm ok with using nibalizer's approach if that's simpler18:02
fungiyep, i don't personally care how we get therwe18:03
fungithere18:03
yolandafungi, but i don't think that he picked the concept: https://review.openstack.org/#/c/189378/2/manifests/file.pp18:04
yolandathe attempt of this patch is to remove files with old naming from logrotate18:04
fungiit's just right now we're still throwing puppet errors on review.openstack.org because of a missing logrotate file, and i would like that if we fix how that's pathed we don't leave extra logrotate configs on people's servers double-rotating some logs18:04
*** rlandy has quit IRC18:05
*** Rockyg has joined #openstack-infra18:05
*** rlandy has joined #openstack-infra18:05
fungiyolanda: seems like your fix is also useful long-term, and his fix is probably better for this transition18:05
yolandamm, that patch he did tries to clean only the /etc/logrotate.d/basename , i'm not sure if that will help18:05
yolandathat's not cleaning the whole old path18:05
AJaegerdevananda: I also pushed these folks in your direction: https://review.openstack.org/#/c/193392/18:06
yolandaso if the old logrotate was /var/log/jeepyb/manage_projects.log, he will clean /etc/logrotate.d/manage_projects.log, not /etc/logrotate.d/var/log/manage_projects.log18:06
yolandathat was the point of that18:06
yolandai'll update the depends-on i created on your change to pont at nibalizer's one, but i think that this needs a fix18:07
*** olaph has quit IRC18:07
greghaynesmordred: ubuntu-minimal really is missing a lot of tools18:08
anteayathe birds are calling, time for me to get a walk in18:08
greghaynesmordred: tools like ping18:08
fungiyolanda: ahh, yeah looking closely i think you're right, though the file_basename var he's inserting seems to be defined outside the scope of that class18:09
yolandahe creates that using basename stdlib method18:09
fungino, i misread18:09
fungiyeah, he's instantiating it in the previous line18:09
yolandabut i think he should remove whole path if exists, not basename18:09
openstackgerrityolanda.robla proposed openstack-infra/puppet-jeepyb: Separate the fetch remotes log dir and file  https://review.openstack.org/19218718:10
yolandajust updated the depends18:10
fungithanks!18:10
*** dannywilson has joined #openstack-infra18:10
*** dannywilson has quit IRC18:10
*** jasondotstar has quit IRC18:10
*** dannywilson has joined #openstack-infra18:10
devanandaAJaeger: yup, thanks. I missed the discussion this morning. the question is similar for both projects, but also different18:11
*** esker has joined #openstack-infra18:12
*** esker has quit IRC18:12
AJaegerdevananda: I guess it  was early this morning for you ;). Discuss with them and figure out what works. For me it looked like something that might fit into ironic...18:12
devanandaAJaeger:  ahc-tools is at least developed by some folks already known in ironic's community (dtantsur is an active core). but it hasn't been discussed yet whether the project-as-a-whole wants to embrace it18:12
*** mrunge has joined #openstack-infra18:12
pleia2fungi: did we want more feedback from cores re: 192473? (git-review drop period in commit message check)18:12
devanandaAJaeger: fwiw, dtantur's explanation as to why this was proposed to stackforge is spot on18:13
yolandafungi, i'll remove my fileremoval part as well, we can move under a new change to simplify it18:13
*** esker has joined #openstack-infra18:13
fungipleia2: i don't think so. because ultimately we aren't specifically dropping that check, we're upgrading to a later version of hacking where the community at large dropped that check18:13
pleia2fungi: yeah, just realized I was reading the old commit message18:14
devanandaAJaeger: in past cycles, i've strongly encouraged folks to do initial work on stackforge and move to openstack/* when the project met a few requirements: it has unit and functional tests, it has sufficient developer docs that other devs can jump in easily, and the project team agrees with the direction18:14
pleia2fungi: thanks18:14
dimsum__pleia2: fungi: can one of you please bless this? (devstack-gate review for oslo.service) https://review.openstack.org/#/c/186365/18:14
devanandaAJaeger, jeblair: I feel that that ^ is still a good bar, but if we're not using stackforge for it, then I'm not sure where to encourage that work to happen, or how to indicate it within the openstack/* namespace18:15
fungidevananda: it's a good question, and i think it still needs some debate. as a thought experiment i like to consider what we'd do if there weren't namespaces at all on these. we still have the governance repo to indicate who's leading the effort for any group of one or more repos18:16
openstackgerrityolanda.robla proposed openstack-infra/puppet-jeepyb: Separate the fetch remotes log dir and file  https://review.openstack.org/19218718:16
yolandaok, that's it18:16
pleia2dimsum__: sure, lgtm18:16
*** pelix has quit IRC18:16
* AJaeger agrees with fungi18:16
dimsum__thanks pleia218:17
openstackgerritMerged openstack-infra/project-config: Take configuraiton control to test hooks for networking-odl  https://review.openstack.org/19243318:17
mgagneclarkb: I'm getting "TypeError: execve() arg 3 contains a non-string value" with nodepool. Remember what was the cause of it?18:17
AJaegerdevananda: right now the default seems to be "go stackforge, then move to openstack". I'd like us to break out of this18:17
devanandafungi: following your thought experiment ... let's say someone creates openstack/ironic-foo, but no one in ironic-core team knows them, so they create a new ironic-foo team in gerrit18:17
yolandajeblair, for nodepool.conf new file, i got some questions that i think they are valid. One is rename nodepool.conf to auth.conf18:18
AJaegerdevananda: If there's a review for ironic-foo in openstack, we ask for which project this is part of.18:18
yolandasecond one is not use [mysql] section for the database, and use the more generic [database] term18:18
*** doug-fish has left #openstack-infra18:18
*** marzif has quit IRC18:18
devanandafungi: do we expect the consumers of openstack (distros, ops, etc) to realize that the openstack/ironic-foo project (the code itself, i mean) is essentially unrelated and not maintained by the same people as openstack/ironic ?18:18
AJaegerIf it's in ironic, we would only approve after your explicit +1 on it.18:18
*** alexpilotti has quit IRC18:18
AJaegerAnd if there's no project adoptiong it, it needs to be a new project -> so first to TC via governance repo before we approve it18:19
devanandaAJaeger: sure, that's how it is today. but i'm following fungi's thought experiment a bit to see what alternatives might exist18:19
AJaegerdevananda: Ah, you think two steps further...18:20
devanandaAJaeger: sure. creation of a new *team* now requires TC approval, whereas an existing team gets a rubber stamp to create a new *repository*18:20
devanandaAJaeger: that's an interesting world, and I largely agree -- the teams are what matters to us developers. but the rest of the world is looking at the software our teams produce18:20
devanandaAJaeger: and they look at the project // code repository names, at least today18:21
fungidevananda: agreed, it's a grey area. this is one of the reasons i'm in favor of jeblair's proposal to require all repos we host to be in tc-recognized projects, and just make it as easy as possible to get a new project recognized by the tc18:21
devanandafungi: s/tc-recognized projects/tc-recognized project teams/18:22
AJaegerdevananda: I see the point. Right now an "ironic-FOO" project in stackforge might tell people "look closely, it's not mature" while in openstack the perception is different. We have the tags in the governance repo but those are new18:22
devanandawe renamed it again :)18:22
fungiyeah18:22
devanandaAJaeger: exactly.18:22
fungiat least then if there is a concern by one project-team over repo names chosen by another project-team, they're both under the governance of the tc and can seek mediation if needed18:22
devanandaAJaeger: also, tags do not indicate maturity of a project, and many folks object to ever allowing that18:23
devanandaAJaeger: so there is no way to tell, from tags, that openstack/ironic-foo and openstack/ironic-bar are of different maturity levels, managed by different teams of people, etc18:23
devanandaso i think the name of a project still carries a ver ysignificant association18:24
fungiaside from the obvious "getting out of the business of renaming constantly repos," i'd love it if the infra team could also get out of the business of deciding who gets to create projects under what teams, with what names, et cetera18:24
devanandafungi: ++18:24
AJaegerfungi: ++18:24
AJaegerdevananda: so that's only part of the OPS tag discussion?18:24
devanandaas an aside, this is one reason why bifrost is called bifrost, not ironic-installer. i anticipate that, some day, someone else may create a similar project using chef or puppet or something else with ironic, and didn't want to imply that there would be one and only one such project18:26
AJaegerdevananda: but nothing hinders the TC to propose a tag "experimental" that the PTL decides on and which would be the default for new repositories...18:26
devanandaAJaeger: heh ... but who decides what project(s) are experimental?18:26
devanandaAJaeger: I think the TC will not do that18:26
AJaegerdevananda: my proposal was: Every new repo is by default experimental. The PTL of the project team adopting it can ask for removal of the flag18:27
persia+118:27
AJaegerdevananda: that puts you into the driver seat for ironic-BAR. Once you consider it non-experimental, you remove the flag.18:28
AJaegerLike today: You experiment in stackforge and when you consider it non-experimental, you ask for moving it to openstack.18:28
fungidevananda: also i'm unsure it matters if a project-team and/or repo is experimental. we have plenty of people creating experimental work and hosting it in our infrastructure already. if we need a registry of what's experimental we can consider maintaining one, but remaning a git repo to indicate whether or not it's experimental is the current thing we would love to _not_ be doing18:28
fungis/remaning/renaming/18:28
devanandafungi: indeed. I support you in wanting to have less of that18:29
mgagneanyone got "TypeError: execve() arg 3 contains a non-string value" when nodepool tries to build an image with DIB? It stopped working for reasons yet unknown to me18:30
greghaynesmgagne: got more detailed traceback?18:31
*** mwagner_afk has joined #openstack-infra18:31
*** jasondotstar has joined #openstack-infra18:32
greghaynesmgagne: also, youre running latest nodepool?18:32
mgagnefrom pypi18:32
mgagnehttp://paste.openstack.org/show/305001/18:32
*** SumitNaiksatam has quit IRC18:32
*** SumitNaiksatam has joined #openstack-infra18:33
*** jsavak has quit IRC18:33
*** jsavak has joined #openstack-infra18:33
fungii tried to pull up one of our dib logs but it has caused firefox to eat all my cpu now18:34
*** kushal has quit IRC18:35
mgagneyea, try chrome instead. AFAIK, firefox doesn't stream the page while it's rendering it unlike chrome18:35
greghaynesmgagne: hrm, that looks like its the qemu-img-options bit18:35
funginot seeing it at http://nodepool.openstack.org/dib.devstack-trusty.log18:35
fungii'll check our debug logs though18:36
mgagneclarkb once saw it in 2014 :D18:36
fungithe internet never forgets!18:36
fungino hits in our nodepool debug logs either18:37
*** kushal has joined #openstack-infra18:37
greghaynesmgagne: youre in a spot where you can edit nodepool and retry?18:37
mgagneyes18:38
greghaynesmgagne: I have some code you could inject to debug18:38
mgagnesure18:38
greghaynesmgagne: add self.log.info(str([type(x) for x in [dib_cmd, img_types, qemu_img_options, filename, img_elements])) right after self.log.info('Running %s' % cmd) in nodepool.py on line 83218:38
greghaynesactually18:38
greghaynesthatll syntax fail18:38
mgagnein _buildImage right?18:38
greghaynesmgagne: self.log.info(str([type(x) for x in [dib_cmd, img_types, qemu_img_options, filename, img_elements]]))18:39
greghaynesyes18:39
*** stevedoor has quit IRC18:40
mgagneI think you were looking for "<type 'unicode'>"  right? :D18:40
mgagnelast argument18:40
mgagneimg_elements18:41
greghaynesnice18:41
mgagnererunning with (what I hope) is now ascii18:42
openstackgerritElizabeth K. Joseph proposed openstack-infra/system-config: Add StevenK and Infra as Zanata admins  https://review.openstack.org/19367718:42
mgagnemy element is "développeur" /jk18:42
greghayneswell, no wonder :p18:42
*** hdd has quit IRC18:43
greghaynesclearly we shouldnt have tried to make the tableflip emoji element18:43
mgagneshould nodepool be updated to str() the value?18:43
greghaynesvery possibly, although that does beg the question of what do we encode as...18:44
greghaynesalmost seems like we should error if it needs to be encoded18:45
mgagnenot sure how encoded that is: https://github.com/openstack-infra/nodepool/blob/ab191ad797a5ae00cc006473cf733c38f92d1ab7/nodepool/cmd/config_validator.py#L9418:46
*** Longgeek has joined #openstack-infra18:46
*** clu_ has quit IRC18:46
*** esker has quit IRC18:46
greghayneshuh, I have no idea if that schema validation accepts unicode for str, I guess it must?}18:46
mgagneor add encoding validation below, I'm really not familiar with voluptuous18:46
greghaynesanyhnow, we should make 100% sure that is the actualy issue before charging ahead18:47
greghaynesI am kind of surprised at it...18:47
mgagneself.log.info('%s of type %s', image.elements, type(image.elements))18:49
mgagnenodepool.DiskImageBuilderThread: ubuntu vm devuser of type <type 'unicode'>18:49
mgagne# file /etc/nodepool/nodepool.yaml18:50
mgagne/etc/nodepool/nodepool.yaml: ASCII text18:50
*** rlandy has quit IRC18:50
*** nihilifer has quit IRC18:51
mgagnenow they are all str and it's crashing still18:53
*** jalvis has joined #openstack-infra18:53
*** ZZelle_ has joined #openstack-infra18:54
*** hodos|2 has joined #openstack-infra18:54
*** HeOS has quit IRC18:56
*** jtomasek has quit IRC18:57
*** hodos has quit IRC18:57
openstackgerritMerged openstack-infra/infra-manual: Improve wording in governance change  https://review.openstack.org/19266519:00
openstackgerritMerged openstack-infra/infra-manual: Add clarification on questions in reviews  https://review.openstack.org/17736419:00
*** jasondotstar has quit IRC19:01
openstackgerritBorneMace proposed openstack-infra/project-config: Made bashate test voting  https://review.openstack.org/19368219:03
*** jalvis has quit IRC19:03
*** sabeen1 has quit IRC19:04
*** mmedvede has quit IRC19:05
greghaynesmgagne: awesome19:05
*** puranamr has quit IRC19:06
mgagneok, I casted everything to str() and it works. now dib requires sudo which it didn't yesterday for some reasons (or sudo got reverted?)19:06
greghaynesdib has always required sodo19:06
greghayneser, sudo19:06
*** puranamr has joined #openstack-infra19:06
mgagneyea so I don't know really what changed here19:06
*** jtomasek has joined #openstack-infra19:07
*** SumitNaiksatam_ has joined #openstack-infra19:08
openstackgerritgreghaynes proposed openstack-infra/nodepool: Cast DIB args to strings  https://review.openstack.org/19368319:09
greghaynesmgagne: ^ is that what you did?19:09
greghaynesalso -> lunch19:09
fungihuh, so we did eventually get our dib uploads, they just started a couple hours later today than yesterday http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=982&rra_id=all19:10
fungimordred: ^19:10
mgagneI think I tried cmd = str(...) and it didn't worked. I cast everything above with str(). will have to test again to make sure19:10
mgagneand I would really like to know what happened since yesterday19:10
*** SumitNaiksatam has quit IRC19:10
*** SumitNaiksatam_ is now known as SumitNaiksatam19:10
openstackgerritMerged openstack-infra/git-review: Update to newer hacking checks  https://review.openstack.org/19247319:10
mordredfungi: yes - I agree that we have just started those19:11
greghaynesmgagne: If you said youre installing from pip then nothing should be different since then - we havent cut any releases19:11
fungimordred: well, they seem to have ended now and nodepool is building new nodes again19:11
mgagnesure, that's why I'm wondering what I did since :-/19:11
*** EricGonczer_ has joined #openstack-infra19:12
mgagnemaybe I paste a string from the internet which was encoded in UTF-8 and now my whole file is considered UTF-8, I'm not sure19:12
mordredfungi: have you looked in the logs to see if we successfully uploaded???19:13
fungimordred: looking now19:13
*** dimtruck is now known as zz_dimtruck19:14
fungimordred: i see devstack-trusty uploads in hpcloud with an age of ~20 minutes19:14
fungithat's reported in the image-list, now i'm correlating to the log19:14
*** sabeen1 has joined #openstack-infra19:15
fungi2015-06-19 17:55:02,014 INFO nodepool.DiskImageUpdater: Uploading dib image id: 123439 from /opt/nodepool_dib/devstack-trusty-1434723361 for devstack-trusty-1434736502.template.openstack.org in hpcloud-b519:16
fungi2015-06-19 18:52:58,124 DEBUG nodepool.DiskImageUpdater: Image id: 123439 saving image 891bf712-579c-4df2-8dd5-471f4e1d15ec19:16
fungi2015-06-19 18:52:58,507 DEBUG nodepool.ProviderManager: Status of image in hpcloud-b5 891bf712-579c-4df2-8dd5-471f4e1d15ec: ACTIVE19:16
fungilooks like that was successful19:17
mordredawesome19:17
*** mmedvede has joined #openstack-infra19:17
mordredany rax hits?19:17
openstackgerritDaneyon Hansen proposed openstack-infra/irc-meetings: Add meeting for Magnum Networking Subgroup  https://review.openstack.org/19369419:17
fungiare we dib-ing anything into rax yet? ubuntu-trusty images yeah?19:17
*** sabeen2 has joined #openstack-infra19:17
mordredyeah19:18
pleia2we're in friday afternoon no meeting time, if anyone wants to look at some channel logging reviews we have piling up: https://review.openstack.org/#/c/161106/ https://review.openstack.org/#/c/187492/ https://review.openstack.org/#/c/190691/ https://review.openstack.org/#/c/193643/19:18
fungithere are ubuntu-trusty images in rax (supposedly) for the past 4.5 hours19:18
fungithat seems incongruous with the upload times for hpcloud19:18
*** hashar has joined #openstack-infra19:18
mordredyah19:18
fungithe same images were uploaded to hpcloud 20 minutes ago19:19
mgagnemuhaha, found it!19:19
*** sdake_ has quit IRC19:19
*** sabeen1 has quit IRC19:20
*** bradjones has quit IRC19:20
mgagneexecve() arg 3 -> http://linux.die.net/man/2/execve int main(int argc, char *argv[], char *envp[])19:20
mgagneenvp <- I added a bunch of env variables for the devuser element. One could be considered a boolean by yaml: yes, true, etc.19:20
fungimordred: though in positive news, the only ubuntu-trusty images reported in rax are from today, so that implies some change in the right direction19:20
*** mrmartin has joined #openstack-infra19:20
mgagnegreghaynes: so I think the correct fix would be to cast all environment variable to str()19:21
*** achanda has quit IRC19:21
greghaynesahh19:21
greghaynesyes, Ill abandon my patch19:21
mgagnethat's why str(cmd) didn't work19:22
mgagnebut I also included env variables when I tried to cast everything19:22
*** hdd has joined #openstack-infra19:22
*** bradjones has joined #openstack-infra19:22
*** bradjones has quit IRC19:22
*** bradjones has joined #openstack-infra19:22
mordredfungi: Last Modified: Fri, 19 Jun 2015 14:22:33 GMT19:23
openstackgerritChris Burroughs proposed openstack-infra/jenkins-job-builder: support sparse/polling path git plugin settings  https://review.openstack.org/19369619:23
mordredfungi: I show that the image uploaded to swift with that as a timestamp19:24
mordredfungi: also, the glance and swift images withteh same name have the same metadata md5 and sha25619:24
mordredso that's nice19:24
fungimordred: excellent. i wonder why the 4-hour time skew19:24
*** cburroughs has joined #openstack-infra19:25
mgagne"Password auth exception. Try number 14..." would be great to know which node is tried ^^'19:25
mordredfungi: hrm. I wonder if the timezone is off on those swift servers19:25
fungiwe did have two spikes today, 4 hours apart, the first higher but finished more quickly19:25
*** kgiusti has left #openstack-infra19:25
openstackgerritMerged openstack-infra/git-review: Fix H238 violation and enable check for that rule  https://review.openstack.org/19247719:26
fungimordred: so i think that nodepool uploaded diskimages into rax shortly after the 14:14 start time, basically first thing, then did other stuff for 4 hours, then uploaded into hpcloud19:27
*** achanda has joined #openstack-infra19:27
fungiit's conceivable with a warm cache that the ubuntu-trusty image build, since there's so little to it, would happen very quickly19:27
fungiand that's the only one we're dib'ing in rax currently19:27
*** Sukhdev has joined #openstack-infra19:28
mordredyah19:28
fungialso i deleted the existing ubuntu-trusty nodes because they were in hpcloud, and the replacement is now building in rax-iad19:28
fungisince ~5 minutes19:28
fungiit disappeared :(19:29
fungilooking in the logs now19:29
mordred2015-06-19 14:01:12,931 INFO nodepool.DiskImageUpdater: Uploading dib image id: 123352 from /opt/nodepool_dib/ubuntu-trusty-1434291359 for ubuntu-trusty-1434722472.template.openstack.org in rax-dfw19:29
mordred2015-06-19 14:49:23,269 INFO nodepool.DiskImageUpdater: Image ubuntu-trusty-1434722472.template.openstack.org in rax-dfw is ready19:29
lifelessmordred: so hu19:29
mordredbtw19:29
lifelesserm19:29
lifelesshi19:29
fungimordred: raise LaunchNetworkException("Unable to find public IP of server")19:29
lifelessactually nvm, I was going to ask for a silly thing because I'm impatient. I'll just do it Monday.19:29
*** mriedem has quit IRC19:30
* mordred hands lifeless a lovely saturday morning19:30
fungilifeless: greetings over there in the future. how's the weekend19:30
*** otter768 has joined #openstack-infra19:30
mordredfungi: yah - I think that's what greghaynes may have been working on yesterday19:30
lifelessfungi: dark and early :)19:30
fungiheh19:30
lifelessseriously. pitch black outside - mid winter 73019:30
*** marcusvrn has quit IRC19:31
fungilifeless: clearly you're wrong. i'm looking out the door and it's a bright, sunny summer day19:31
fungiwe can't both be right19:31
fungimordred: so anyway, we _can_ dib into rax now, but we can't boot the result (yet)19:31
fungiso, anyway, yay progress!19:32
openstackgerrityolanda.robla proposed openstack-infra/nodepool: Allow to delete nodes with missing target or provider  https://review.openstack.org/19262919:32
greghaynesmordred: fungi: yep, poking at that19:32
mordredfungi: it's SO MUCH PROGRESS19:33
greghaynesfungi: it boots, just no networking19:33
mordredfungi: now all we have to do is fix an element19:33
fungiso much progress it all over yuo screen19:33
fungiand mine too19:33
* jeblair lunches19:33
fungii'm going back to doc sprinting if anyone needs me19:34
*** otter768 has quit IRC19:34
mordredgreghaynes: anything I can do to help on that - I feel like I have not been useful in helping to diagnose19:35
*** _nadya_ has joined #openstack-infra19:35
*** _nadya_ has quit IRC19:36
greghaynesmordred: dont think so, although soon I may be throwing some theories your way, have to test a couple thoughts19:36
mordredcool19:36
*** ihrachyshka has joined #openstack-infra19:36
greghaynesmordred: most of the time is just how slow dev cycles are for buil/upload/boot19:36
*** _nadya_ has joined #openstack-infra19:36
openstackgerritMerged openstack-infra/git-review: Ignore .eggs cruft  https://review.openstack.org/19247819:37
*** jtomasek has quit IRC19:37
*** jasondotstar has joined #openstack-infra19:37
greghaynesoh, funny bug - I think if we run gean -i eth0, then glean -i eth1 we end up with only eth1 into in e/n/i19:37
mordredgreghaynes: hrm. maybe it's a race with the event-driven stuff?19:38
fungiespecially if you ran gean rather than glean for the first one19:38
greghaynesyep, thats kind of what I suspect19:38
*** jasondotstar has quit IRC19:38
*** pc_m has quit IRC19:38
greghaynesadded a bunch of debug and am respinning19:38
mordredwoot19:39
*** jsavak has quit IRC19:39
openstackgerritMerged openstack-infra/system-config: Add app-catalog-admin mailing list  https://review.openstack.org/19147819:39
*** jsavak has joined #openstack-infra19:39
lifelessfungi: lets take the mean19:40
*** EricGonc_ has joined #openstack-infra19:41
*** amotoki_ has quit IRC19:41
mordredlifeless: what is the mean of gean and glean?19:42
greghaynesglear19:42
mordredwe need a glear project clearly19:42
persiaghchg19:42
*** puranamr has quit IRC19:44
*** puranamr has joined #openstack-infra19:45
lifelessmordred: summer and winter, day and night.19:45
*** EricGonczer_ has quit IRC19:45
*** jasondotstar has joined #openstack-infra19:46
*** flepied has joined #openstack-infra19:46
*** hashar has quit IRC19:46
fungimordred: or glare19:46
mgagnemordred: we had someone come to an interview and talked about the OpenStack Glare and Redstone projects19:46
*** jasondotstar has quit IRC19:47
fungihah19:47
*** yamahata has joined #openstack-infra19:48
*** yamahata has quit IRC19:48
*** yamahata has joined #openstack-infra19:49
openstackgerritMerged openstack-infra/devstack-gate: Add oslo.service to devstack-vm-gate-wrap.sh  https://review.openstack.org/18636519:49
*** marzif has joined #openstack-infra19:49
*** jsavak has quit IRC19:50
timrcmgagne, Hope you hired them on the spot.19:50
mgagnetimrc: so we are the first to contribute to this Glare project19:50
openstackgerritSean Dague proposed openstack-infra/project-config: don't run dsvm tests on doc only changes  https://review.openstack.org/19370719:53
timrcSeriously though... keystone, pbr... whoever is naming these projects has poor taste in beer ;-)19:53
greghayneskeystone-lite, even19:53
sdagueso, anyone want to give me a friday gift and land https://review.openstack.org/193707 so we can land devstack doc patches faster :)19:54
timrcWe should switch Openstack codenames over to cheap beers... (M)ilwuake's Best, (N)atty Light, (S)chlitz, etc.19:54
*** jasondotstar has joined #openstack-infra19:54
*** puranamr has quit IRC19:55
*** puranamr has joined #openstack-infra19:55
*** mrmartin has quit IRC19:57
anteayaso if I contribute to Glare to I get sunglasses as a gift?20:01
anteayapolarized?20:01
dimsum__Glare and Redstone. whoa20:02
anteayaRedstone is a lake close to where I live20:03
*** achanda has quit IRC20:03
*** dustins has quit IRC20:04
*** boris-42 has joined #openstack-infra20:04
*** mrmartin has joined #openstack-infra20:07
*** stevemar has joined #openstack-infra20:07
greghaynesmordred: so whats happening is glean is getting run before the config-drive is findable via blkid20:08
greghaynesmordred: so were back in init dependency hell20:08
mordredgreghaynes: OH GOOD20:10
*** krtaylor has quit IRC20:10
*** peristeri has quit IRC20:10
greghaynesmordred: oh wait, I think im wrong, I put my debug info wrong20:11
mordredPHEW20:11
greghaynesits telling me no confi drive found but it might actualyl mean the opposite20:11
*** e0ne has quit IRC20:11
*** mrmartin has quit IRC20:12
greghaynesmordred: another bug found though - we depend on openssh-server which isnt in ubuntu-minimal20:13
*** dan is now known as Guest8780820:14
*** marzif has quit IRC20:14
*** dan_ has joined #openstack-infra20:14
greghaynesbecause we do the dpkg-reconfigure openssh-server with set -e20:14
sdagueAJaeger, anteaya, fungi - thanks!20:14
*** dan_ is now known as Guest9629920:14
anteayasdague: happy Friday to you20:14
mordredgreghaynes: heh. well, also - we should definitely put an ssh server on there20:16
*** sdake has joined #openstack-infra20:16
*** sdake_ has joined #openstack-infra20:17
anteayado we have a server up for hound currently?20:17
anteayaI'm guessing kdc01 and 02 are not it20:18
fungianteaya: there's a hound.openstack.org which is up for testing, mentioned in the spec20:18
anteayaand I can't browse to them anyway20:18
anteayaah I didn't see it in cacti20:18
fungianteaya: kdc01 and 02 are kerberos domain controllers20:18
anteayadidn't check the spec, thanks20:19
anteayaah okay thanks20:19
fungianteaya: er, kerberos key distribution centers, sorry20:19
*** tonytan4ever has quit IRC20:19
openstackgerritMerged openstack-infra/project-config: don't run dsvm tests on doc only changes  https://review.openstack.org/19370720:20
anteayanp20:20
anteayaI would have to look up the difference anyway20:20
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Clean up old SSH keys for fungi  https://review.openstack.org/19371320:20
*** sdake has quit IRC20:21
anteayataron: something of note, in hound when I search for `git pack ` I still get results with git package though I did specify the whitespace after pack in my entry20:21
* taron notes20:22
anteayathanks20:22
anteayaand git package is all I am returned20:22
taronif I'm going to work on refreshing the config files I might as well hack on the rest of it too20:22
anteayano git pack results at all, regardless of flags20:22
taronhuh20:22
mordredtaron, anteaya: I'm not sure what their definition of regex is20:23
mordredbut I can't seem to get any regexes to work20:23
mordredanteaya: I _DO_ get some resulst for git pack with no quotes20:23
*** tlian2 has joined #openstack-infra20:23
anteayaI didn't use quotes20:23
anteayahttp://hound.openstack.org/?q=git%20pack&i=nope&files=&repos=20:23
taronyeah the results without quotes are all git package it seems20:24
anteayataron: thanks for confirming20:24
fungianteaya: you want git repack20:24
mordredthis works: http://hound.openstack.org/?q=git.*pack\b.*&i=nope&files=&repos=20:24
anteayaah20:24
fungibut yeah20:24
*** miqui has quit IRC20:24
mordredhttp://hound.openstack.org/?q=git%20repack\b&i=nope&files=&repos=20:24
mordredanteaya: \b is a regex char that says "word boundary"20:24
mordredfwiw20:24
mordredand yeah - regexes _do_ work when I search for something with hits :)20:25
taronmy kingdom for consistent regex implementation across languages/tools :|20:25
mordredtaron: it seems fairly consistent with what I'd expect ... my first tries just weren't actually anything with hits20:25
* anteaya must attend to the stove for about 10 minutes and will return20:25
jeblairif only there were a, i don't know, "perl compatible regular expression library" that any language could link against and use!20:25
*** tlian has quit IRC20:25
taron\b seems to be one of the more consistently supported characters20:26
fungianteaya: http://hound.openstack.org/?q=git.*repack&i=nope&files=&repos= manages to get the places i expect to see (system-config, puppet-gerrit, zuul)20:26
*** prad_ has quit IRC20:26
fungijeblair: we should invent that and call it pcre20:26
jeblairfungi: good name for it!20:26
taronheh20:26
fungimainly because we're uninventive people20:26
fungiwhen it comes to names20:26
mordredfungi: and then reimplement it in scratch in languages that don't link against C20:26
mordreds/in/from/20:27
fungimordred: sometimes very badly20:27
fungiand inconsistently20:27
* taron uses pcre for everything they can but doesn't know what to do with themself when bits of it are missing20:27
fungiand that brings us up to... today20:27
taronyup20:27
openstackgerritMerged openstack-infra/shade: Add Neutron/Nova Floating IP list/search/get  https://review.openstack.org/18349620:27
* mordred is not sure if taron means "taron" or "pcre" by "it"20:28
taronpcre20:28
* mordred is not implying that taron misspoke - but rather that choosing to be confused is more fun20:28
taronheh20:28
taronthis is fair20:28
jeblairmordred always choses confusion20:28
jeblairi think20:28
jeblairmaybe?20:28
jeblairsometimes20:28
jeblairmordred always causes confusion20:28
mordredjeblair: I may or may not always, sometimes or never choose confusion or lack thereof20:28
*** wenlock has quit IRC20:29
Clintseems like a useful strategy20:29
*** puranamr has quit IRC20:29
*** puranamr has joined #openstack-infra20:29
* mordred thinks his policy is too clearly stated20:29
fungimordred embodies the chaos and confusion which underlies the creative process20:30
*** bswartz has quit IRC20:30
*** esker has joined #openstack-infra20:30
openstackgerritEmilien Macchi proposed openstack-infra/project-config: puppet-gnocchi: enable beaker jobs  https://review.openstack.org/19371520:30
*** marzif has joined #openstack-infra20:30
*** rossella_s has quit IRC20:30
*** rossella_s has joined #openstack-infra20:31
*** julim has quit IRC20:31
* mordred underlies the embodiement of confusion and creativity in the chaos process20:31
anteayamordred fungi thank you for the search queries both were better than mine20:32
taronchaos... blood for the blood god?20:32
fungithe sacred chao20:33
Clintchao of chaos20:34
fungias a member of the golden apple corps, i fully endorse this consistent inconsistency20:34
mordredanteaya: fun with regexes is fun20:35
* taron is fond of regex crosswords20:35
mordredanteaya: hopefully soon you can get mmmpork to mind-meld some perl with you, at which point you'll never quite be the same20:35
taronI'm quite bad at them, but they're good practice20:36
mordredtaron: that sounds wow20:36
anteayamordred: I'm looking forward to it?20:36
*** zz_dimtruck is now known as dimtruck20:36
mordredanteaya: exactly20:36
taronhttp://regexcrossword.com/ relatedly20:36
anteayawonderful20:37
*** jasondotstar has quit IRC20:37
*** marzif has quit IRC20:38
*** mmedvede has quit IRC20:38
fungioh, and since it's friday, i should partake of no hot dog buns. almost forgot20:39
openstackgerritK Jonathan Harker proposed openstack-infra/project-config: Fix logic error in gate-project-config-projects-yaml  https://review.openstack.org/19371720:39
mordredtaron: ok. that is entirely too much fun20:40
taron:)20:40
jesusaurustaron: oh cool20:40
*** CaptainMorgan is now known as morgan20:41
jesusaurusmordred: fungi: jeblair: I found a lack of test coverage in check_valid_gerrit_projects.py ^^20:41
*** _nadya_ has quit IRC20:41
*** amotoki_ has joined #openstack-infra20:42
*** _nadya_ has joined #openstack-infra20:42
*** cburroughs has quit IRC20:43
*** dprince has quit IRC20:43
fungijesusaurus: great catch. i take it that job still succeeds on the current dataset with that fix20:43
jesusaurusyep :)20:44
fungiahh, so it does. i see the job has already completed20:45
fungilooks like someone added the label check later20:45
fungiand missed that they were adding after an unrelated short-circuit20:46
*** _nadya_ has quit IRC20:47
*** amotoki_ has quit IRC20:47
*** mrunge has quit IRC20:47
*** vnod has quit IRC20:47
*** unicell has joined #openstack-infra20:47
*** ihrachyshka has quit IRC20:47
*** marzif has joined #openstack-infra20:51
openstackgerritK Jonathan Harker proposed openstack-infra/project-config: Add 'head-reference' as a valid gerrit project label  https://review.openstack.org/19372120:51
*** tlian2 has quit IRC20:51
*** unicell has quit IRC20:54
*** rlandy has joined #openstack-infra20:56
*** unicell has joined #openstack-infra20:57
*** esker has quit IRC20:58
openstackgerritMonty Taylor proposed openstack-infra/shade: Use accessIPv4 if it's there  https://review.openstack.org/19356920:58
openstackgerritMonty Taylor proposed openstack-infra/shade: Add IPv6 to the server information too  https://review.openstack.org/19358120:58
SpamapSjeblair: oh interesting, I was just looking through gear and I noticed another thing the ambiguity around unique ids affected.21:00
*** mjturek1 has quit IRC21:00
*** marzif has quit IRC21:00
SpamapSjeblair: In other gearman clients, when you submit a job, they use a hash ring to select which server based on the unique id.21:00
SpamapSjeblair: this way if you have coalescing, you are more likely to coalesce only once.21:00
* anteaya having finished dinner clicks on http://regexcrossword.com/21:02
openstackgerritDoug Hellmann proposed openstack-infra/infra-manual: Update gerrit ACL docs to mention library-release  https://review.openstack.org/19330921:04
*** rfolco has quit IRC21:05
*** soren has joined #openstack-infra21:08
*** krtaylor has joined #openstack-infra21:08
*** mmedvede has joined #openstack-infra21:09
openstackgerritMerged openstack-infra/project-config: Fix logic error in gate-project-config-projects-yaml  https://review.openstack.org/19371721:09
*** radez is now known as radez_g0n321:14
*** tvanderwiel has quit IRC21:18
*** hashar has joined #openstack-infra21:19
*** bradjones has quit IRC21:20
openstackgerritDaneyon Hansen proposed openstack-infra/irc-meetings: Add meeting for Magnum Networking Subgroup  https://review.openstack.org/19369421:22
*** bradjones has joined #openstack-infra21:22
*** bradjones has quit IRC21:22
*** bradjones has joined #openstack-infra21:22
openstackgerritMerged openstack/requirements: Fix setup.cfg.  https://review.openstack.org/19292121:25
*** amitgandhinz has quit IRC21:26
*** aysyanne has quit IRC21:27
greghaynesmordred: pretty sure whats happening is just the bug where if we run glean -i <interface> we overwrite existing interface config, and were running eth0, then eth1, eth0 is the public and gets overwritten therefore no networking21:27
*** Sukhdev has quit IRC21:27
greghaynesmordred: I kind of think we should be using the e/n/i.d rather then e/n/i then this isnt an issue21:28
*** bswartz has joined #openstack-infra21:28
fungigreghaynes: how is that handled for red hat derivative distros?21:29
*** tqtran is now known as tqtran_afk21:29
greghaynesfungi: looks like we do per-interface21:29
greghayneser, per-interface files21:30
fungigot it, so separate /etc/sysconfig/network-scripts/ifcfg-eth0 et cetera21:30
*** Rockyg has quit IRC21:30
greghaynesyep21:30
fungiso this was already fine for them, just not debian derivatives21:31
*** otter768 has joined #openstack-infra21:31
fungithat makes sense21:31
greghaynesyea, and im guessing what happened is we switched to triggering this init script per-interface rather than once for all interfaces, this bug cropped21:31
greghayneser, this bug cropped up then21:31
*** dboik has quit IRC21:33
waynranteaya: great, and I was starting to actually feel productive today21:34
mordredgreghaynes: cool. let's do ifcfg.d21:34
anteayawaynr: ah congratulations?21:34
waynr(was being sarcastically appreciative of your link)21:35
greghaynesmordred: ya, working on that now21:35
anteayawaynr: ah, sarcasm floats right past me, grew up with a household of it21:35
*** otter768 has quit IRC21:35
anteayawaynr: glad you are appreciative, taron is the original sharer of the link21:35
mordredgreghaynes: woot. I guess that probably means a change to both glean and the simple-init element (such that simple-init should lay down /e/n/i with the include line in it)21:36
dhellmannfungi, mordred : could one of you look at this change to sync requirements into mox3? https://review.openstack.org/#/c/190741/21:36
*** tqtran_afk is now known as tqtran21:38
greghaynesmordred: yep21:38
mordreddhellmann: +321:39
dhellmannmordred: ty21:39
anteayamordred: I want you to find a restaurant with a triclinium: https://en.wikipedia.org/wiki/Triclinium21:39
hasharlifeless: thanks a ton for diskimage-builder , seems you are one of the early committer :-}21:39
anteayaI had been looking up fainting couches and it led me here21:39
anteayamordred: I don't have to be in attendance at the restaurant I just need to know one exists21:40
*** dboik has joined #openstack-infra21:43
*** justchecking has joined #openstack-infra21:45
*** ldnunes has quit IRC21:46
*** esker has joined #openstack-infra21:51
*** stevemar has quit IRC21:52
*** dboik has quit IRC21:53
*** esker has quit IRC21:54
openstackgerritDaneyon Hansen proposed openstack-infra/irc-meetings: Add meeting for Magnum Networking Subgroup  https://review.openstack.org/19369421:55
bknudsonrequirements update breaks keystone unit tests -- https://review.openstack.org/#/c/190405/721:56
*** esker has joined #openstack-infra21:57
*** marzif has joined #openstack-infra21:58
*** soren has quit IRC21:58
openstackgerritColleen Murphy proposed openstack-infra/irc-meetings: Update the chair for the Puppet OpenStack meetings  https://review.openstack.org/19373821:59
*** erlon has quit IRC22:00
*** xyang1 has quit IRC22:00
*** abitha has quit IRC22:03
*** abitha has joined #openstack-infra22:03
*** justchecking has quit IRC22:04
*** justchecking has joined #openstack-infra22:05
*** dimtruck is now known as zz_dimtruck22:09
*** marzif has quit IRC22:10
*** isq has joined #openstack-infra22:11
*** ddieterly has quit IRC22:12
*** puranamr has quit IRC22:13
*** yamahata has quit IRC22:14
*** RichardRaseley has quit IRC22:14
*** rlandy has quit IRC22:18
*** dimsum__ has quit IRC22:19
*** rlandy has joined #openstack-infra22:19
*** dimsum__ has joined #openstack-infra22:20
*** puranamr has joined #openstack-infra22:22
*** puranamr_ has joined #openstack-infra22:24
*** puranamr_ has quit IRC22:24
*** sdake_ has quit IRC22:25
hashareveryone must have left but. Nodepool more or less works for Wikimedia Fundation !!! https://integration.wikimedia.org/ci/computer/ci-jessie-wikimedia-38/ :-D22:26
hasharhave a good week-end everone22:26
*** hdd has quit IRC22:27
*** puranamr has quit IRC22:27
*** bknudson has quit IRC22:30
*** jasondot_ has quit IRC22:32
anteayahashar: yay22:32
anteayahashar: and to you too22:32
*** dimsum__ has quit IRC22:32
hasharof course nodepool deleted the slave as soon as I ran a job on it :-}22:32
anteayaof course22:33
anteaya:)22:33
jeblairhashar: it has a habit of doing that.  :)  and cool!22:33
anteayaworking as expected22:33
hasharoverall it was not that hard to install and setup22:33
anteayanice22:33
hasharhad to understand what OpenStack is and all the related jargon  and of course read the doc22:33
*** marzif has joined #openstack-infra22:33
hasharso kudos guys for supporting unrelated 3rd parties with nice software and docs :-]22:34
jeblairhashar: we try to confuse people by writing docs.  no one expects that.22:35
*** EricGonc_ has quit IRC22:35
hasharoh22:35
hasharI just amend them as needed22:35
jeblaireven better :)22:35
jeblairfungi: when you have a minute, it looks like we still have 2 ask.o.o servers; do you know if we're ready to delete one?22:36
jeblair(for that matter, we also have two review.o.o servers.  i bet we could delete one of them as well)22:37
*** puranamr has joined #openstack-infra22:37
*** gyee has quit IRC22:39
hasharhappy week-end.22:39
*** amotoki_ has joined #openstack-infra22:43
jeblairnovaclient.exceptions.Forbidden: Policy doesn't allow compute_flavor:create:image_backed to be performed. (HTTP 403) (Request-ID: req-66be769b-2901-4ab0-b263-664ad7a35812)22:44
*** hashar has quit IRC22:45
jeblairoh that's neat; that happens if you try to use launch-node to create a compute-flavor instance in rax22:45
*** puranamr has quit IRC22:47
openstackgerritMerged openstack-infra/system-config: Add jhesketh to infrastructure servers  https://review.openstack.org/19087922:47
*** amotoki_ has quit IRC22:49
openstackgerritMerged openstack-infra/system-config: Up the haproxy timeout for git-fe*  https://review.openstack.org/19264922:49
*** puranamr has joined #openstack-infra22:50
*** puranamr has quit IRC22:54
*** puranamr has joined #openstack-infra22:54
*** hdd has joined #openstack-infra22:58
*** SotK has quit IRC22:59
*** SotK has joined #openstack-infra22:59
*** Swami has quit IRC23:00
openstackgerritMerged openstack/requirements: sync requirements to mox3  https://review.openstack.org/19074123:03
uberjayasselin: huh, something funny going on for me. changes which have deps on unknown projects are sitting in the queue and not getting executed. see screenshot: http://i.imgur.com/rvnHzfa.png23:04
*** zigo has quit IRC23:05
*** marzif has quit IRC23:06
*** ondergetekende has quit IRC23:06
*** juggler has quit IRC23:06
*** juggler has joined #openstack-infra23:06
*** dangers is now known as dangers_away23:06
*** puranamr has quit IRC23:06
*** zigo has joined #openstack-infra23:07
*** ondergetekende has joined #openstack-infra23:07
*** taron has quit IRC23:07
*** thedodd has quit IRC23:08
*** puranamr has joined #openstack-infra23:09
*** jgrimm is now known as zz_jgrimm23:10
*** juggler has quit IRC23:16
*** juggler has joined #openstack-infra23:17
*** tbarron has quit IRC23:19
*** MarkAtwood has quit IRC23:19
*** krtaylor has quit IRC23:21
*** eharney has quit IRC23:21
*** jlvillal has quit IRC23:21
*** ianw has quit IRC23:21
*** eharney has joined #openstack-infra23:22
*** krtaylor has joined #openstack-infra23:22
*** vaidy has quit IRC23:25
*** hdd has quit IRC23:26
*** isviridov_away has quit IRC23:27
anteayaI always feel so bad standing up23:27
anteayathe cat has spend such time to get comfy23:27
anteayaand then I stand up and all is lost23:27
anteaya:(23:27
*** esker has quit IRC23:28
*** esker has joined #openstack-infra23:28
fungijeblair: yeah, pretty sure we can delete the review and ask servers which don't match dns at this point. just a sec and i'll get rid of the right ones23:29
*** fedexo has quit IRC23:31
*** otter768 has joined #openstack-infra23:32
*** romcheg has left #openstack-infra23:32
*** fedexo has joined #openstack-infra23:32
fungijeblair: all gone!23:32
*** jlvillal has joined #openstack-infra23:33
*** tlian has joined #openstack-infra23:33
*** vaidy_ril has joined #openstack-infra23:33
*** isviridov_away has joined #openstack-infra23:33
*** ianw has joined #openstack-infra23:34
*** tbarron has joined #openstack-infra23:34
fungiback to trying to make myself document things on a friday night23:36
*** otter768 has quit IRC23:36
jeblairfungi: i'm still trying to figure out how to launch a node that boots from a volume :(23:37
fungijeblair: that sounds like friday fun for masochists23:38
jeblairfungi: i'm starting to think it involves setting up an ec2 account23:38
fungihttps://artfulanxiety.files.wordpress.com/2015/06/pdvd_056.jpg23:39
*** ajmiller has quit IRC23:42
*** SumitNaiksatam has quit IRC23:45
*** hogepodge has quit IRC23:45
*** EricGonczer_ has joined #openstack-infra23:46
anteayaha ha ha23:47
*** rlandy has quit IRC23:47
fungiit's the american way23:48
anteayaeffective in many cases23:49
anteayain canada every business has to have at least three purposes23:49
anteayathe dollar store/post office/drycleaners is in the centre of town23:49
*** taron has joined #openstack-infra23:51
*** hemna_ has joined #openstack-infra23:52
*** hemna has quit IRC23:53
anteayacinder channel is fairly quiet right now23:56
anteayaI looked to see if someone was about23:57
*** ZZelle_ has quit IRC23:57
*** dannywilson has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!