| mordred | anteaya: Ithought at first you were intending on logstashing your new fridge | 00:01 |
|---|---|---|
| mordred | andI know IoT is a thing | 00:01 |
| mordred | but still | 00:01 |
| mordred | I think logstash for your new fridge is excessive | 00:01 |
| clarkb | zer0c00l I believe its running qemu VMs via virsh with no virt | 00:01 |
| clarkb | mordred could be useful for knowing when to buy eggs | 00:02 |
| jesusaurus | or the frequency of being out of milk? | 00:02 |
| mordred | clarkb: wouldn't your fridge have to know that first? | 00:02 |
| pleia2 | mordred: I thought the same, fridge that tweets, must keep logs... | 00:02 |
| pleia2 | but then I thought better of it :) | 00:02 |
| mordred | pleia2: :) | 00:03 |
| pleia2 | because for one, the chances of an api to do such a thing... :D | 00:03 |
| mordred | now - graphite to track the temp of the fridge | 00:03 |
| pleia2 | I want a fridge with an api | 00:03 |
| pleia2 | right | 00:03 |
| anteaya | mordred: logstashing my fridge would be exciting | 00:03 |
| openstackgerrit | Merged openstack-infra/nodepool: Document SIGINT / SIGUSR2 https://review.openstack.org/203571 | 00:03 |
| clarkb | mordred they keep threatening rfids on everything | 00:03 |
| anteaya | I do not want a fridge with an api | 00:03 |
| anteaya | nor an ice maker | 00:03 |
| mordred | aww. ice makers are nice | 00:03 |
| jesusaurus | there might also be a form of ocr for common fridge items | 00:03 |
| anteaya | don't want to listen to them | 00:03 |
| mordred | ah | 00:03 |
| mordred | yes | 00:03 |
| mordred | that would be an issue | 00:04 |
| clarkb | ice makers are nice until the feed pops off and fllods the kitchen | 00:04 |
| mordred | I enjoy being able to get crushed ice from a fridge | 00:04 |
| anteaya | getting rid of this one because it never stops running | 00:04 |
| mordred | clarkb: ++ | 00:04 |
| pleia2 | ice for cocktails is very important | 00:04 |
| *** jaypipes has quit IRC | 00:04 | |
| anteaya | clarkb: yup, fridges with no icemakers don't do that | 00:04 |
| mgagne | can't get enough of those "Hash Sum mismatch" from Ubuntu mirrors :-/ | 00:04 |
| pleia2 | anteaya: even my modern fridge has an ambient noise to it with the ice maker (also the not so ambient "I made an ice cube!") | 00:04 |
| anteaya | pleia2: given my hermit status taht is just +1 more for no icemakers | 00:04 |
| *** ccrouch has quit IRC | 00:04 | |
| clarkb | mgagne its a feature of apt repos :/ | 00:05 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Debug message leads to (n*m) log growth. https://review.openstack.org/246684 | 00:05 |
| anteaya | it is the I made an ice cube I am not interested in | 00:05 |
| *** ccrouch has joined #openstack-infra | 00:05 | |
| mgagne | clarkb I didn't get that much in the past, I'm wondering what changed... | 00:05 |
| *** akshai_ has quit IRC | 00:05 | |
| clarkb | mgagne it would be tied to frequency of package updates | 00:05 |
| pleia2 | fwiw, there have been a lot of ubuntu updates this past week | 00:05 |
| mtreinish | mordred, clarkb, fungi, SpamapS: I started a list of the steps I think we'll need to do for the big subunit2sql migration: https://etherpad.openstack.org/p/the-big-subunit2sql-migration | 00:06 |
| anteaya | pleia2: nice | 00:06 |
| pleia2 | (I keep track of it for the ubuntu weekly newsletter, it was high) | 00:06 |
| mtreinish | let me know if you think I missed anything (which is very likely) | 00:06 |
| anteaya | wonderful | 00:06 |
| *** maiteb has joined #openstack-infra | 00:06 | |
| mgagne | it's a daily occurence for us, to a point where we ended up mirroring our own mirror so it doesn't update that often | 00:06 |
| mordred | mtreinish: step 12 is awesome | 00:06 |
| mordred | mgagne: we have a design for a mirror that will not ever have those errors, but we have not gotten to it yet | 00:07 |
| mtreinish | mordred: it's the most important one :) | 00:07 |
| mordred | mgagne: once we do, we'll hook you up :) | 00:07 |
| anteaya | mtreinish: fixed a typo | 00:08 |
| *** rbrndt_ has quit IRC | 00:09 | |
| mordred | pleia2, clarkb: what's the email I should use for signing up for new cloud accounts for infra again? | 00:09 |
| *** bhunter71 has joined #openstack-infra | 00:09 | |
| mordred | also- what should I use for phone number? clarkb's cell phone? | 00:09 |
| mtreinish | anteaya: thanks | 00:09 |
| openstackgerrit | Merged openstack/requirements: Updated from generate-constraints https://review.openstack.org/245504 | 00:10 |
| clarkb | mordred please no :) | 00:10 |
| pleia2 | def clarkb's cell | 00:10 |
| anteaya | mtreinish: doing the least I can do :) | 00:10 |
| pleia2 | mordred: I'd just look through our aliases in our password file, maybe infra-root@? | 00:10 |
| pleia2 | (but infra-root is the canonical one) | 00:11 |
| clarkb | I think we used review@ for hp and rax | 00:11 |
| pleia2 | interesting choice | 00:11 |
| clarkb | for phone it should be something they can call because they often do call and ask if you are spamming | 00:11 |
| mordred | I'll start with mine, we can update it if/when we have a good answer | 00:12 |
| *** amuller has quit IRC | 00:13 | |
| clarkb | maybe google voice to forward to all of us? | 00:13 |
| clarkb | (sorry jeblair fungi I have not better ideas) | 00:13 |
| *** derekh has joined #openstack-infra | 00:13 | |
| pleia2 | unless we can figure out something with asterisk | 00:14 |
| pleia2 | but still need a phone provider (which is what google is) | 00:14 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql: Stop attempting to run a mysqldump before migrations https://review.openstack.org/246686 | 00:14 |
| pleia2 | google voice bothers me less than other solutions for that reason | 00:14 |
| pleia2 | s/other solutions/other proprietary solutions for out things | 00:15 |
| pleia2 | our | 00:15 |
| pleia2 | typing | 00:15 |
| clarkb | what we need is a robot assistant | 00:16 |
| anteaya | that makes ice | 00:17 |
| openstackgerrit | Merged openstack-infra/puppet-graphite: Set statsd to delete idle counters/timers https://review.openstack.org/246610 | 00:18 |
| pleia2 | lol | 00:18 |
| clarkb | named crawford (too much fallout4) | 00:18 |
| openstackgerrit | Merged openstack-infra/puppet-cgit: Fix logic in selinux execs https://review.openstack.org/203868 | 00:18 |
| clarkb | interesting Gerrit list claims that HEAD is replicated if you change it via the web ui | 00:19 |
| *** stack_ has joined #openstack-infra | 00:19 | |
| clarkb | (not true for our gerrit version iirc) | 00:19 |
| *** otter768 has joined #openstack-infra | 00:20 | |
| *** derekh has quit IRC | 00:20 | |
| *** ddieterly has joined #openstack-infra | 00:22 | |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Add generic logstash manifests https://review.openstack.org/199790 | 00:23 |
| openstackgerrit | Sachi King proposed openstack-infra/project-config: Add non-voting constraints jobs to nova https://review.openstack.org/219582 | 00:24 |
| *** ashleighfarnham has quit IRC | 00:25 | |
| *** otter768 has quit IRC | 00:25 | |
| openstackgerrit | Steve McLellan proposed openstack-infra/project-config: Add python-searchlightclient repo https://review.openstack.org/246691 | 00:26 |
| *** dhp_ is now known as dhp | 00:27 | |
| *** sergmelikyan has joined #openstack-infra | 00:29 | |
| jeblair | we do have a phone provider, and a phone number | 00:29 |
| pleia2 | jeblair: oh? | 00:29 |
| jeblair | +1 512-808-5750 | 00:29 |
| jeblair | the conference server :) | 00:30 |
| *** yamamoto has joined #openstack-infra | 00:30 | |
| clarkb | zaro: https://docs.oracle.com/javase/tutorial/networking/urls/creatingUrls.html maybe useful for fixing the openid bug | 00:30 |
| jeblair | it's not helpful if you want to reach a person | 00:30 |
| clarkb | jeblair: there isn't a mechanism for ya that | 00:30 |
| jeblair | but there's no need to use google voice | 00:30 |
| clarkb | I know when I signed up for rax at least they called me | 00:30 |
| jeblair | just to get a telephone number | 00:30 |
| clarkb | so we need something that ends up in front of a human | 00:31 |
| clarkb | jeblair: it also forwards calls to N other numbers iirc | 00:31 |
| clarkb | is that somethin we can convince asterisk to do for us? | 00:31 |
| jeblair | jhesketh: regarding my earlier point about mock spreading: doesn't mocking a fake sound like exactly that? https://review.openstack.org/206043 | 00:32 |
| openstackgerrit | Steve McLellan proposed openstack-infra/project-config: Add python-searchlightclient repo https://review.openstack.org/246691 | 00:32 |
| *** maiteb has quit IRC | 00:32 | |
| jeblair | jhesketh: that could all be done with a small enhancement to the existing fakes, and then we could exercise it in the normal code paths, right? | 00:33 |
| jeblair | clarkb: of course. i'm not sure that i want what we're talking about though. :) | 00:34 |
| jeblair | i think our providers should just shut us off if there's a problem and send us email. | 00:34 |
| clarkb | that also wfm | 00:34 |
| clarkb | they wanted to know what I would use my account for | 00:35 |
| jeblair | (if we _did_ set up telco forwarding, i would probably have that asterisk call my asterisk so it can just go into the bitbucket) | 00:35 |
| jeblair | (er, i mean, voicemail) | 00:35 |
| pleia2 | hah | 00:35 |
| pleia2 | a voicemail box that emails us when someone calls and leaves a message would be tolerable for me | 00:36 |
| clarkb | I am very much a noob when it comes to telephony. Tmobile gives me unlimited data and good international roaming | 00:36 |
| clarkb | thats what matters right? | 00:36 |
| pleia2 | I don't like voice calls, I don't often don't answer them | 00:37 |
| pleia2 | clarkb: heh, yeah, data was all I had in japan, it was fine :) | 00:37 |
| jeblair | pleia2: yeah, that's what my home asterisk box does with most calls | 00:37 |
| jeblair | i love calls from people i want to talk to. cloud providers are not on the list. | 00:38 |
| clarkb | I hav emy phone set to drop calls not from contacts during night hours and otherwise its easy enoguh to answer or not | 00:38 |
| clarkb | also I am aamzed it took ansdroid so long to add that feature | 00:38 |
| clarkb | "I don't want calls at 3am kthxbai" | 00:38 |
| clarkb | one time hpcloud called me at 3am, this feature was not yet avialable | 00:40 |
| *** Sukhdev has quit IRC | 00:41 | |
| clarkb | ok, tomorrow will be pick up logstash 2.0 work. Hopefully we get through the rules and have a working set to use | 00:42 |
| clarkb | then its redo puppet (maybe with some of my newfound logstash testers) | 00:42 |
| clarkb | then fix the multiline stuff by deploying logstash 2.0 | 00:43 |
| nibalizer | pleia2: will do when i get home (a few minutes) | 00:43 |
| pleia2 | nibalizer: great, thanks :) | 00:43 |
| *** Sukhdev has joined #openstack-infra | 00:43 | |
| *** Sukhdev has quit IRC | 00:44 | |
| *** thorst has joined #openstack-infra | 00:44 | |
| *** yamamoto has quit IRC | 00:48 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Add wheel_mirror_slaves that use pypi-mirror https://review.openstack.org/165240 | 00:48 |
| docaedo | I'm setting a manifest to pip install the app_catalog (as cloned by git) into a virtual environment - is the python::pip module the right one to use (https://github.com/stankevich/puppet-python)? | 00:48 |
| *** sergmelikyan has quit IRC | 00:48 | |
| *** mestery has quit IRC | 00:50 | |
| *** thorst has quit IRC | 00:51 | |
| zaro | yah! power restored | 00:51 |
| *** thorst has joined #openstack-infra | 00:51 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/meetbot: Add warning if starting a meeting in an unrecognised channel https://review.openstack.org/246695 | 00:52 |
| docaedo | and that question is a prelude to - can anyone tell me how to make that module pip-install from a local file rather than pypi? | 00:52 |
| pleia2 | zaro: glad to hear it :) | 00:52 |
| *** jyuso1 has joined #openstack-infra | 00:53 | |
| *** thorst has quit IRC | 00:56 | |
| openstackgerrit | Sachi King proposed openstack-infra/zuul: Set basepython to python2.7 globally https://review.openstack.org/246697 | 00:56 |
| *** bhunter71 has quit IRC | 00:56 | |
| * docaedo once again trots out his puppet ignorance... | 00:56 | |
| zaro | clarkb: it can be fixed but still cannot explain why our gerrit+openid config returns empty token so how to explain the fix? | 00:57 |
| *** armax has joined #openstack-infra | 00:57 | |
| clarkb | zaro: well they ar etwo completely different bugs | 00:57 |
| anteaya | zaro: yay power | 00:57 |
| *** gyee has quit IRC | 00:57 | |
| clarkb | zaro: so they should be treated as such, conflating them will just make it harder to debug and fix | 00:57 |
| nibalizer | pleia2: sending now | 00:57 |
| anteaya | tonyb: I was actually just looking at that, thanks for the credit | 00:58 |
| nibalizer | pleia2: do we have a date that we are gonna try it | 00:58 |
| tonyb | anteaya: np As I said it was inspired by you change. | 00:59 |
| *** Kennan has quit IRC | 00:59 | |
| tonyb | anteaya: I would have reused that change ID but I wasn't able to restore it | 00:59 |
| pleia2 | nibalizer: nope | 00:59 |
| anteaya | tonyb: my concern is we will again be redirected to extend the config, have you looked at the supybot config? I fell in there last time and was happy to come out alive | 00:59 |
| anteaya | tonyb: no worries | 00:59 |
| lifeless | Nakato: when the project-config patches are up; please cc me on them? | 00:59 |
| anteaya | tonyb: thanks for looking at it, kind of you to credit me :) | 00:59 |
| tonyb | anteaya: Well I added acomment to say I will do the supybot config thing but I need to learn how to do it | 01:00 |
| pleia2 | nibalizer: I'm hoping it just gets bumped a couple days, but we still don't have a firm solution ironed out so it's hard to say | 01:00 |
| anteaya | tonyb: yeah, I can't figure it out | 01:00 |
| *** sputnik13 has quit IRC | 01:00 | |
| *** sergmelikyan has joined #openstack-infra | 01:00 | |
| nibalizer | pleia2: https://etherpad.openstack.org/p/gerit_downtime_delay sound ok? | 01:00 |
| Nakato | lifeless: Can do, (you mean co contrib right?) | 01:00 |
| anteaya | don't commit to learning supybot until you look at it | 01:00 |
| anteaya | tonyb: the project is old, no activity in 5 years | 01:01 |
| tonyb | anteaya: I looked at it it doesn't look too terrible. | 01:01 |
| anteaya | tonyb: the homepage is a stackforge page with no documentation I can find | 01:01 |
| *** sputnik13 has joined #openstack-infra | 01:01 | |
| pleia2 | nibalizer: short and sweet, lgtm (and I'd still reply to the announcement email) | 01:01 |
| zaro | clarkb: i would fix it by checking for null token and not adding extra '/'. i think that is missing in gerrit code. do you agree? | 01:01 |
| anteaya | tonyb: did you find any docs for it? | 01:01 |
| tonyb | anteaya: I created a venv/mocjk env for testing the meetbot plugin | 01:01 |
| clarkb | zaro: no, I think you need to join the urls properly always | 01:01 |
| anteaya | okay | 01:01 |
| tonyb | anteaya: I had the code written 2 hours ago but I wanted to test it ;P | 01:01 |
| clarkb | zaro: regardless of the token situation | 01:01 |
| nibalizer | pleia2: ok | 01:02 |
| anteaya | tonyb: makes sense | 01:02 |
| tonyb | anteaya: nothing helpful so I was goign to fallback to reading the code and *lots* of testing. | 01:02 |
| anteaya | well maybe your code will get accepted | 01:02 |
| anteaya | you have helpful error messages | 01:03 |
| * tonyb crosses fingers. | 01:03 | |
| *** Kennan has joined #openstack-infra | 01:03 | |
| lifeless | Nakato: no, I just mean link me as a reviewer | 01:03 |
| lifeless | Nakato: so I get emailed on the discussion is all | 01:03 |
| *** sflanigan has quit IRC | 01:03 | |
| zaro | clarkb: the token can be different depending on entry point and where you should get redirected to after login. | 01:03 |
| Nakato | lifeless: Will do. | 01:03 |
| anteaya | tonyb: these are the only docs I found: https://github.com/Supybot/Supybot/tree/master/docs | 01:03 |
| lifeless | Nakato: I haven't contributed to the patch, wouldn't make sense to have my name on it IMO :) | 01:03 |
| clarkb | zaro: exactly thats why it should always be handled correctly | 01:04 |
| zaro | flashing lights is not a good sign | 01:04 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Create ansible-role-jobs for JJB and zuul https://review.openstack.org/246633 | 01:04 |
| anteaya | tonyb: none tell me anything about the various config settings, they expect you to set config via a wizard while running the code | 01:04 |
| Nakato | lifeless: Also I'll point at https://review.openstack.org/246697 Sick of fighting that every time I try and run tox on project-config | 01:04 |
| *** hemna is now known as hemnafk | 01:04 | |
| crinkle | greghaynes: tests pass on https://review.openstack.org/#/c/165240/ now but will need clarkb or nibalizer to sanity check | 01:04 |
| crinkle | because i don't actually know how mirrors or slaves work | 01:04 |
| *** rlandy has quit IRC | 01:04 | |
| Nakato | er, whoops, didn't realise it was checking py26... | 01:05 |
| zaro | clarkb: yes, agreed. but that's a very generic solution. i think the code is trying to do that. probably missing a few things though. | 01:05 |
| *** mriedem has joined #openstack-infra | 01:05 | |
| *** shashank_hegde has quit IRC | 01:05 | |
| tonyb | anteaya: Yeah its defined in code which is good and bad | 01:05 |
| anteaya | tonyb: not helpful for me | 01:06 |
| zaro | clarkb: well so checking for null token will fix it for us. so that's one if we get merged upstream will work. | 01:06 |
| anteaya | the only other thing I could think of was to create an ircmeeting/localMeetingConfig.py file as directed here: http://git.openstack.org/cgit/openstack-infra/meetbot/tree/ircmeeting/meeting.py?id=5f8bdbeef15f9ab7486e2f2f8e7c622a223cdb4e#n47 | 01:07 |
| *** aeng has quit IRC | 01:07 | |
| *** rossella_s has quit IRC | 01:07 | |
| *** rossella_s has joined #openstack-infra | 01:07 | |
| tonyb | anteaya: Yeah I think that's the right way to do it. | 01:08 |
| zaro | alright, me is signing out for today. had a long week. cyas | 01:08 |
| tonyb | anteaya: and then drive the values in there from puppet | 01:08 |
| anteaya | nibalizer: I see the notice you sent, thank you | 01:08 |
| anteaya | tonyb: I think that is config but keeps us away from supybot config | 01:09 |
| * anteaya shudders | 01:09 | |
| jhesketh | jeblair: (sorry had an appointment) yeah, I can see your point.. mocking a fake doesn't make a lot of sense | 01:09 |
| anteaya | tonyb: but I had figured out what to put in that file yet | 01:09 |
| tonyb | anteaya: Oh I thought supybot sourced that for us | 01:09 |
| anteaya | tonyb: I think so, it should | 01:09 |
| *** jamielennox is now known as jamielennox|away | 01:10 | |
| anteaya | I had originally looked a supybot config to see if I could add a setting there | 01:10 |
| anteaya | so many settings | 01:10 |
| tonyb | anteaya: :) | 01:10 |
| anteaya | tonyb: there is a self.specialChannels for the testing channels for meetbot which don't have logs written for them | 01:11 |
| anteaya | http://git.openstack.org/cgit/openstack-infra/meetbot/tree/ircmeeting/meeting.py?id=5f8bdbeef15f9ab7486e2f2f8e7c622a223cdb4e#n154 | 01:11 |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-openstackci: Update logstash/elastic search from system-config https://review.openstack.org/246701 | 01:12 |
| jeblair | beef! | 01:12 |
| *** serg_melikyan has joined #openstack-infra | 01:12 | |
| jeblair | 5f8bdbeef15f9ab7486e2f2f8e7c622a223cdb4e ~~~~ | 01:12 |
| jeblair | ~~~~ | 01:12 |
| anteaya | ha ha ha | 01:12 |
| anteaya | ha ha ha | 01:12 |
| anteaya | words in shas bingo | 01:13 |
| pleia2 | lol | 01:13 |
| *** sergmelikyan has quit IRC | 01:13 | |
| greghaynes | crinkle: sweet ty | 01:13 |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-openstackci: Update logstash/elastic search from system-config https://review.openstack.org/246701 | 01:14 |
| jeblair | nibalizer, clarkb, greghaynes, SpamapS: do you think we might be able to do some simple nodepooly things against infra-cloud (like i've been doing with bb and ovh)? even though it's not the final hardware, that seems in the spirit of 'continue to poke at current setup to learn/advance as much as we can'... | 01:15 |
| clarkb | crinkle: ^ | 01:15 |
| jeblair | crinkle: er, ^ | 01:15 |
| tonyb | anteaya: Yeah but that doesn't seem to be a config oprtion | 01:15 |
| *** ildikov has quit IRC | 01:15 | |
| *** sflanigan has joined #openstack-infra | 01:15 | |
| *** sflanigan has joined #openstack-infra | 01:15 | |
| jeblair | sorry, i miscompleted c<tab> | 01:15 |
| *** ildikov has joined #openstack-infra | 01:16 | |
| nibalizer | jeblair: i was heading that direction last week | 01:16 |
| crinkle | jeblair: it's really close | 01:16 |
| nibalizer | didn't get there though | 01:16 |
| nibalizer | for instance just getting a few nodes and doing the 'how long does devstack take' would be awesome I think | 01:17 |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/system-config: Remove logstash/elasticsearch classes, use puppet-openstackci https://review.openstack.org/240011 | 01:18 |
| anteaya | tonyb: hmmm | 01:19 |
| crinkle | right now i'm trying to get static ips working for the nodes, which is tricky because glean doesn't totally work and bifrost doesn't exactly support the dnsmasq configuration that would let it dhcp properly | 01:19 |
| *** sputnik13 has quit IRC | 01:19 | |
| anteaya | tonyb: M is meeting and has a channel var: http://git.openstack.org/cgit/openstack-infra/meetbot/tree/ircmeeting/meeting.py?id=5f8bdbeef15f9ab7486e2f2f8e7c622a223cdb4e#n742 | 01:20 |
| anteaya | tonyb: maybe we can test the value of M.channel against something? | 01:22 |
| *** thorst has joined #openstack-infra | 01:23 | |
| *** thorst has quit IRC | 01:23 | |
| *** stevemar_ has quit IRC | 01:23 | |
| *** thorst has joined #openstack-infra | 01:23 | |
| anteaya | tonyb: and other option: http://git.openstack.org/cgit/openstack-infra/meetbot/tree/ircmeeting/meeting.py?id=5f8bdbeef15f9ab7486e2f2f8e7c622a223cdb4e#n664 | 01:23 |
| *** aeng has joined #openstack-infra | 01:23 | |
| anteaya | so meeting.py seems to know what channel it is in | 01:23 |
| *** thorst has quit IRC | 01:24 | |
| openstackgerrit | Merged openstack-infra/puppet-logstash: Look for curator in both /usr/bin and /usr/local/bin https://review.openstack.org/246525 | 01:24 |
| *** Sree has joined #openstack-infra | 01:24 | |
| *** thorst has joined #openstack-infra | 01:25 | |
| *** Sree has joined #openstack-infra | 01:25 | |
| *** oomichi has joined #openstack-infra | 01:25 | |
| tonyb | Hmm let me try something | 01:26 |
| * anteaya lets tonyb try something | 01:28 | |
| lifeless | Nakato: maybe we don't need py26 for zuul anymore ? | 01:28 |
| lifeless | jeblair: ^ | 01:28 |
| lifeless | jeblair: does infra run or care about py26 support for zuul ? | 01:28 |
| lifeless | jeblair: (context is https://review.openstack.org/#/c/246697/) | 01:28 |
| *** Swami has quit IRC | 01:28 | |
| clarkb | uh | 01:29 |
| clarkb | that change in general does not make sense to me | 01:29 |
| *** thorst has quit IRC | 01:29 | |
| clarkb | we have python version specific envs | 01:29 |
| *** Sree has quit IRC | 01:29 | |
| clarkb | py26, py27, py34 etc | 01:29 |
| clarkb | so why both setting basepython? | 01:29 |
| lifeless | clarkb: tox runs sdist outside the venv | 01:29 |
| lifeless | clarkb: ubuntu has 'tox' actually installed with python3 | 01:30 |
| lifeless | clarkb: so you run tox, and then it uses a python3 virtualenv and python3 interpreter to make the virtualenv and create the sdist | 01:30 |
| *** aeng has quit IRC | 01:30 | |
| lifeless | clarkb: for projects that aren't python3 ready, this is bad, it breaks :) | 01:30 |
| *** aeng has joined #openstack-infra | 01:30 | |
| clarkb | then your ubuntu is broken | 01:30 |
| lifeless | clarkb: it doesn't affect whats *inside* the venvs, naturally | 01:30 |
| clarkb | you should be pip installing tox | 01:30 |
| lifeless | clarkb: welcome to Ubuntu | 01:30 |
| clarkb | (maybe from another venv) | 01:30 |
| lifeless | clarkb: I recommend that, but this does no harm | 01:31 |
| clarkb | except it breaks not py27 | 01:31 |
| lifeless | clarkb: how so ? | 01:31 |
| clarkb | that test failed | 01:31 |
| clarkb | I am just going off of the change you linked me | 01:31 |
| lifeless | clarkb: so, yes, its a hardcoded basepython | 01:31 |
| *** oomichi has quit IRC | 01:31 | |
| lifeless | clarkb: on machines with *no* python 2.7, everythingw will fail | 01:31 |
| lifeless | clarkb: but it doesn't break !py27, you can still run e.g. py34 etc | 01:32 |
| clarkb | then why did py26 fail? | 01:32 |
| *** ivar-laz_ has joined #openstack-infra | 01:32 | |
| clarkb | (because there is no py27?) | 01:32 |
| lifeless | because the forced basepython doesn't exist on that host | 01:32 |
| gus | (I should have pushed https://review.openstack.org/#/c/160230/ further than neutron) | 01:32 |
| clarkb | ok I wouldn't assume that py27 is everywhree then | 01:33 |
| lifeless | because there is no python2.7 (which is distinct from py27) | 01:33 |
| clarkb | instead you should maybe set basepython to python | 01:33 |
| clarkb | (and arch can fail) | 01:33 |
| *** jaypipes has joined #openstack-infra | 01:33 | |
| lifeless | Nakato: is 'python' on your system 3.x or 2.x ? | 01:33 |
| clarkb | lifeless: according to python it must be 2.x | 01:33 |
| clarkb | lifeless: arch breaks this | 01:33 |
| clarkb | I do not care aout arch being broken | 01:33 |
| clarkb | gus: thats slightly differnet (but related) | 01:34 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql: Stop attempting to run a mysqldump before migrations https://review.openstack.org/246686 | 01:34 |
| *** ivar-lazzaro has quit IRC | 01:36 | |
| *** yamamoto has joined #openstack-infra | 01:37 | |
| *** ivar-laz_ has quit IRC | 01:37 | |
| Nakato | lifeless: 3.x | 01:37 |
| clarkb | Nakato: are you on arch? | 01:38 |
| clarkb | this is important because arch explicitly breaks the rules here | 01:38 |
| *** ivar-lazzaro has joined #openstack-infra | 01:38 | |
| *** pradk has quit IRC | 01:38 | |
| clarkb | and as a result I don't think we should bend over backwrads to accomodate that | 01:38 |
| *** eharney has quit IRC | 01:38 | |
| Nakato | clarkb: yes, and yes that pep | 01:38 |
| *** ivar-lazzaro has quit IRC | 01:38 | |
| *** ivar-lazzaro has joined #openstack-infra | 01:39 | |
| Nakato | I don't think it means we shouldn't fix the problems either. | 01:39 |
| tonyb | anteaya: I have it reading the the 2 optiosn from global config but supybot can't change them dynamically | 01:39 |
| clarkb | I suggest you take it up with your distro | 01:39 |
| tonyb | anteaya: looking at that now. | 01:39 |
| clarkb | Nakato: there is no sane way to fix the problems | 01:39 |
| clarkb | Nakato: they have intentionally decided not to accomodate any problem fixes | 01:39 |
| Nakato | Arch pre-dates the pep | 01:39 |
| *** ivar-lazzaro has quit IRC | 01:39 | |
| clarkb | they can still fix it | 01:39 |
| anteaya | tonyb: if we put something in here and called it self.M.channel and compared it to a value set in a config file? | 01:40 |
| anteaya | http://git.openstack.org/cgit/openstack-infra/meetbot/tree/ircmeeting/meeting.py?id=5f8bdbeef15f9ab7486e2f2f8e7c622a223cdb4e#n310 | 01:40 |
| Nakato | Also it just means enforcing some sane defaults in pep instead of expecting dev systems to run py27. You'd see similar breakges on systems that run py26 as a default | 01:40 |
| *** ivar-lazzaro has joined #openstack-infra | 01:40 | |
| *** paco20151113 has joined #openstack-infra | 01:40 | |
| clarkb | Nakato: no, my suggestion would fix that | 01:41 |
| clarkb | but it won't work on arch | 01:41 |
| clarkb | but thats why I suggest it because it will work everywhere else | 01:41 |
| *** shashank_hegde has joined #openstack-infra | 01:41 | |
| *** zhenguo has joined #openstack-infra | 01:42 | |
| anteaya | tonyb: by global config do you mean this? http://git.openstack.org/cgit/openstack-infra/puppet-meetbot/tree/templates/meetingLocalConfig.py.erb | 01:42 |
| tonyb | anteaya: I think doing it in plugin.py is fine it's just a question (IIUC) of using meeting.Config.New_Option rather than hard coding the list in the code. | 01:42 |
| tonyb | anteaya: currently I'm using http://git.openstack.org/cgit/openstack-infra/meetbot/tree/ircmeeting/meeting.py?id=5f8bdbeef15f9ab7486e2f2f8e7c622a223cdb4e#n45 | 01:43 |
| tonyb | but that may be wrong as it's not being reloaded when I tell supybot to do that thing | 01:43 |
| *** bhunter71 has joined #openstack-infra | 01:44 | |
| anteaya | hmmm | 01:44 |
| anteaya | I agree with your direction | 01:44 |
| anteaya | and I think once we figure out the value we put it in here: http://git.openstack.org/cgit/openstack-infra/puppet-meetbot/tree/templates/meetingLocalConfig.py.erb | 01:45 |
| anteaya | or did I misunderstand you? | 01:45 |
| clarkb | (I think basepython is actually python already) | 01:46 |
| *** ccrouch has quit IRC | 01:46 | |
| *** tlian2 has joined #openstack-infra | 01:46 | |
| clarkb | Nakato: would it work for you to make a virtualenv that you use where python is python2.x properly instead? | 01:46 |
| clarkb | honestly thats likely path of least resistance and how mtreinish does it iirc | 01:46 |
| mtreinish | clarkb: oh, arch pain | 01:47 |
| mtreinish | Nakato: yeah as python devs using arch linux it's a burden we have to carry :( | 01:47 |
| *** annegentle has quit IRC | 01:47 | |
| Nakato | clarkb: Actually it doesn't with project-config | 01:47 |
| clarkb | Nakato: even if you source it ahead of time? | 01:47 |
| Nakato | clarkb: Does not work. | 01:48 |
| clarkb | Nakato: maybe thats a bug in project-config then and not in zuul's tox.ini? | 01:48 |
| Nakato | clarkb: This is why I actually care about fixing this one, usually that's what I do when I hit a problem because it's not worth this arguemnt. | 01:48 |
| *** tiswanso has joined #openstack-infra | 01:48 | |
| Nakato | tox calls bash which calls zuuls tox | 01:48 |
| *** tiswanso has quit IRC | 01:49 | |
| *** zhurong has joined #openstack-infra | 01:49 | |
| clarkb | but bash should inherit the calling env? | 01:49 |
| clarkb | oh except that tox filters that | 01:49 |
| clarkb | so maybe thats the real bug | 01:49 |
| *** tlian has quit IRC | 01:49 | |
| *** tiswanso has joined #openstack-infra | 01:49 | |
| *** dkranz has joined #openstack-infra | 01:49 | |
| *** boris-42 has joined #openstack-infra | 01:50 | |
| clarkb | Nakato: if you have tox pass through the env does that work? | 01:51 |
| *** ivar-lazzaro has quit IRC | 01:52 | |
| *** otter768 has joined #openstack-infra | 01:53 | |
| lifeless | that makes sense to me | 01:53 |
| Nakato | clarkb: I think it's run-layout.sh that's dropping the env. Because tox normally passes PATH though. | 01:55 |
| Nakato | clarkb: #!/bin/bash ends up with a clean env right? | 01:55 |
| tonyb | anteaya: Ah okay I'm not mocking out the correct config stuff that's why the dynamic relaod isn't working. | 01:55 |
| anteaya | tonyb: ah ha, knew you would figure it out | 01:55 |
| clarkb | Nakato I can never remember and wouldnhave to read the manpage | 01:56 |
| clarkb | I know it looks to bashrc but not bash login or something | 01:57 |
| *** Liuqing has joined #openstack-infra | 01:58 | |
| *** paco20151113 has quit IRC | 01:58 | |
| jroll | anyone mind an easy "drop 2.6" +A? https://review.openstack.org/#/c/246514/ | 01:58 |
| anteaya | jroll: done | 01:59 |
| *** mestery has joined #openstack-infra | 02:00 | |
| jroll | thanks anteaya :) | 02:00 |
| anteaya | pleia2: if the reschedule the upgrade conversation comes up before next meeting, I'm out Friday morning through Sunday night | 02:00 |
| anteaya | jroll: welcome | 02:00 |
| *** sflanigan has quit IRC | 02:01 | |
| *** aeng has quit IRC | 02:01 | |
| pleia2 | anteaya: noted, thanks | 02:02 |
| openstackgerrit | Merged openstack-infra/project-config: Drop Python 2.6 tests for python-ironicclient https://review.openstack.org/246514 | 02:02 |
| *** baoli has joined #openstack-infra | 02:03 | |
| *** shashank_hegde has quit IRC | 02:05 | |
| *** apoorvad has quit IRC | 02:07 | |
| *** tqtran has quit IRC | 02:08 | |
| *** bpokorny_ has joined #openstack-infra | 02:09 | |
| *** bardia has quit IRC | 02:11 | |
| *** bpokorny has quit IRC | 02:12 | |
| *** sflanigan has joined #openstack-infra | 02:13 | |
| *** sflanigan has joined #openstack-infra | 02:13 | |
| *** bpokorny_ has quit IRC | 02:13 | |
| *** stevemar_ has joined #openstack-infra | 02:14 | |
| *** apoorvad has joined #openstack-infra | 02:15 | |
| *** aeng has joined #openstack-infra | 02:15 | |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 02:15 |
| *** EricGonczer_ has joined #openstack-infra | 02:17 | |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 02:18 |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 02:19 |
| *** _amrith_ is now known as amrith | 02:21 | |
| *** unicell1 has quit IRC | 02:22 | |
| openstackgerrit | Christopher Aedo proposed openstack-infra/puppet-apps_site: Add python WSGI server https://review.openstack.org/245315 | 02:24 |
| *** baoli_ has joined #openstack-infra | 02:25 | |
| *** baoli has quit IRC | 02:28 | |
| *** sdake has joined #openstack-infra | 02:28 | |
| *** apoorvad has quit IRC | 02:28 | |
| *** davideagnello has quit IRC | 02:28 | |
| *** davideagnello has joined #openstack-infra | 02:29 | |
| *** dims_ has quit IRC | 02:30 | |
| *** zz_dimtruck is now known as dimtruck | 02:31 | |
| *** annegentle has joined #openstack-infra | 02:38 | |
| *** EricGonczer_ has quit IRC | 02:40 | |
| *** stevemar_ has quit IRC | 02:42 | |
| *** fawadkhaliq has joined #openstack-infra | 02:42 | |
| *** Yanyanhu has joined #openstack-infra | 02:42 | |
| *** yonglihe has quit IRC | 02:42 | |
| *** stevemar_ has joined #openstack-infra | 02:42 | |
| *** dkranz has quit IRC | 02:44 | |
| *** ddieterly has quit IRC | 02:46 | |
| *** annegentle has quit IRC | 02:46 | |
| *** bcourt has quit IRC | 02:47 | |
| *** tlian2 has quit IRC | 02:50 | |
| *** camunoz has quit IRC | 02:53 | |
| *** Liuqing has quit IRC | 02:55 | |
| *** Liuqing has joined #openstack-infra | 02:55 | |
| *** gnuoy has quit IRC | 02:55 | |
| *** jamespage has quit IRC | 02:56 | |
| *** thorst has joined #openstack-infra | 02:57 | |
| *** thorst has quit IRC | 02:57 | |
| *** bcourt has joined #openstack-infra | 02:59 | |
| *** NobodyCam has quit IRC | 02:59 | |
| *** NobodyCam has joined #openstack-infra | 03:00 | |
| *** pfalleno1 has quit IRC | 03:01 | |
| *** pfallenop has joined #openstack-infra | 03:03 | |
| *** pfallenop has joined #openstack-infra | 03:03 | |
| *** mestery has quit IRC | 03:03 | |
| *** ddieterly has joined #openstack-infra | 03:05 | |
| *** amrith is now known as _amrith_ | 03:05 | |
| *** ddieterl_ has joined #openstack-infra | 03:06 | |
| *** rfolco has joined #openstack-infra | 03:08 | |
| *** rick_ has joined #openstack-infra | 03:09 | |
| *** ddieterly has quit IRC | 03:10 | |
| *** maishsk has quit IRC | 03:10 | |
| *** aukhan has joined #openstack-infra | 03:11 | |
| *** yuanying_ has joined #openstack-infra | 03:17 | |
| *** yuanying has quit IRC | 03:19 | |
| *** Sree has joined #openstack-infra | 03:20 | |
| *** Sree has quit IRC | 03:21 | |
| *** rfolco has quit IRC | 03:21 | |
| *** yuanying_ has quit IRC | 03:21 | |
| *** Sree has joined #openstack-infra | 03:21 | |
| *** Sree has quit IRC | 03:22 | |
| * notmorgan waves at -infra | 03:22 | |
| *** tlian has joined #openstack-infra | 03:22 | |
| * Clint squints at notmorgan | 03:22 | |
| notmorgan | Clint: oh sorry, I'm kindof far away. | 03:23 |
| * Clint adjusts the focus. | 03:23 | |
| * notmorgan WAVES LOUDLY (yet loud waving) at CLINT | 03:23 | |
| notmorgan | yes* | 03:23 |
| Clint | good job | 03:23 |
| *** rhallisey has quit IRC | 03:23 | |
| *** Sree has joined #openstack-infra | 03:24 | |
| notmorgan | ooh. it's like 7:24PM and dark here. this whole more north thing is wierd ;) | 03:24 |
| Clint | it's been dark here since like 5pm | 03:24 |
| notmorgan | Clint: oh same here. just noticing how dark it is atm | 03:24 |
| docaedo | hey, long as you guys are waving at each other, have a second to help explain how I might call setup.py from a puppet manifest and make it install something in a virtual env? | 03:24 |
| *** achanda has joined #openstack-infra | 03:24 | |
| notmorgan | docaedo: hm. related to ^ those patches? | 03:25 |
| docaedo | I bet one of you knows what I'm doing wrong here :) | 03:25 |
| docaedo | notmorgan: yes | 03:25 |
| notmorgan | hm.. | 03:25 |
| notmorgan | well the issue i've seen is that setup.py kindof sucks to call, i've been told repeatedly to not call it directly, always use pip | 03:25 |
| *** _amrith_ is now known as amrith | 03:25 | |
| docaedo | line 113 of https://review.openstack.org/#/c/245315/4/manifests/init.pp is my hassle | 03:25 |
| *** calebb has quit IRC | 03:25 | |
| docaedo | yes | 03:25 |
| docaedo | I would use python::pip gladly if it would install something from a local source | 03:26 |
| notmorgan | it will. | 03:26 |
| notmorgan | pip install ./<path> | 03:26 |
| notmorgan | oh puppet thing might not | 03:26 |
| notmorgan | ? | 03:26 |
| docaedo | ok yes - but the python::pip module (which is aware of virtualenvs) does not let you do that | 03:26 |
| notmorgan | docaedo: sounds like a bug in the python::pip module :( lame | 03:27 |
| Clint | i think you should bribe crinkle to fix it | 03:27 |
| docaedo | I would call pip install from an exec if I could make it do something in the virtual env, but it doesn't seem to | 03:27 |
| notmorgan | ah easy to call the right pip | 03:27 |
| notmorgan | you know the venv path right? | 03:27 |
| notmorgan | so <venv>/bin/pip | 03:27 |
| notmorgan | use that. | 03:27 |
| docaedo | notmorgan: yep, tried that, doesn't work (will run again real quick to fetch exact error | 03:28 |
| notmorgan | ok | 03:28 |
| *** tjones has joined #openstack-infra | 03:28 | |
| *** tjones has left #openstack-infra | 03:28 | |
| *** sekrit has quit IRC | 03:28 | |
| * crinkle has been fighting with puppet all day, it would have to be a very large bribe | 03:29 | |
| *** tsg has joined #openstack-infra | 03:30 | |
| notmorgan | crinkle: hehe | 03:30 |
| * docaedo could probably expense an appropriate sized bribe for this! | 03:30 | |
| docaedo | http://paste.openstack.org/show/479200/ | 03:30 |
| notmorgan | docaedo: i could probably chip in for an appropriate sized <this is not really a bribe cause that is against policy> | 03:31 |
| * notmorgan looks at the paste. | 03:31 | |
| docaedo | right, no, not a bribe .. a uh, thank you gift! | 03:31 |
| notmorgan | oooooh | 03:31 |
| notmorgan | are you working in a git repo? | 03:31 |
| docaedo | yes | 03:31 |
| notmorgan | this is the wierd PBR tbing... | 03:32 |
| notmorgan | have you *ever* tagged something in this repo? | 03:32 |
| docaedo | well it is again, but now in a brand new place | 03:32 |
| docaedo | the PBR thing was solved finally by two things: | 03:32 |
| docaedo | proper path for wsgi, and updating the source so the package name matched the directory (changed from catalog to openstack_catalog) | 03:32 |
| docaedo | then everything worked, and I was very happy, so very happy | 03:33 |
| notmorgan | opt/apps_site is that the right repo? | 03:33 |
| docaedo | then I though 'well now, better make the manifest do all the things' and I am now sad, so very sad | 03:33 |
| docaedo | correct, that's the repo | 03:33 |
| notmorgan | hmmmmm. | 03:33 |
| notmorgan | ooh | 03:35 |
| *** sdake has quit IRC | 03:35 | |
| notmorgan | wonder if this is related to how pip is installed. | 03:35 |
| docaedo | I think maybe if I could get puppet to essentially do the same thing as "source venv/bin/activate" before trying this command, it would work, but it doesn't... | 03:35 |
| *** sekrit has joined #openstack-infra | 03:35 | |
| notmorgan | docaedo: i don't tbink it would change this error | 03:35 |
| docaedo | I get the same error if I do setup.py install | 03:36 |
| notmorgan | docaedo: you'd still run into it. my guess is distribute or setuptools is wonky in the VENV | 03:36 |
| notmorgan | or whateve that is supposed to be today. i can't remember | 03:36 |
| docaedo | ah! I saw a swift bug about that once | 03:36 |
| notmorgan | yeah | 03:36 |
| notmorgan | it's hit a lot of the python-*clients too | 03:36 |
| docaedo | pip install --upgrade setuptools is supposed to fix it | 03:37 |
| docaedo | let for me to try | 03:37 |
| notmorgan | remember to do it in a way that the VENV is affected | 03:37 |
| docaedo | yep, trying just from the venv in shell and says it's already up-to-date :/ | 03:37 |
| docaedo | maybe I'm wrong about python::pip not being able to do what I want? Can you see a way to tell it to install from local filesystem path here? https://github.com/stankevich/puppet-python/blob/master/manifests/pip.pp | 03:40 |
| notmorgan | crinkle: ^? | 03:40 |
| notmorgan | docaedo: my guess is you can use URL | 03:40 |
| notmorgan | and do file:// | 03:40 |
| *** binbincong_ has quit IRC | 03:41 | |
| notmorgan | bah | 03:41 |
| notmorgan | https://github.com/stankevich/puppet-python/blob/master/manifests/pip.pp#L159 | 03:41 |
| notmorgan | no you can't | 03:41 |
| mordred | docaedo: what you trying to do? | 03:41 |
| notmorgan | if it's a local git repo you could possibly do it | 03:41 |
| notmorgan | mordred: he's trying to install with puppet python::pip from local fielsystem | 03:42 |
| mordred | interesting | 03:42 |
| nibalizer | whats the code that runs it? | 03:42 |
| nibalizer | oh its in scrollback | 03:42 |
| *** tsg has quit IRC | 03:43 | |
| mordred | we install a few things in virtualenvs | 03:43 |
| *** dimtruck is now known as zz_dimtruck | 03:43 | |
| mordred | oh - but we do that outside of puppet | 03:44 |
| docaedo | I can recap as needed, because if you lot can't figure this out, it's unpossible | 03:44 |
| *** akshai has joined #openstack-infra | 03:44 | |
| notmorgan | mordred: yeah. and he's havint issues with PBR version error exception when using the venv pip in exec | 03:44 |
| mordred | docaedo: what's the command you're trying to run? | 03:44 |
| notmorgan | ooh | 03:45 |
| docaedo | command => "/opt/virtenv-app-catalog/bin/pip install ${root_dir}" | 03:45 |
| mordred | yah. that will run pip inside the venv | 03:45 |
| docaedo | or same thing except python setup.py install | 03:45 |
| *** sdake has joined #openstack-infra | 03:45 | |
| mordred | which is the same as doing activate first | 03:45 |
| docaedo | you would think! but then I get this error: http://paste.openstack.org/show/479200/ | 03:46 |
| *** tlian has quit IRC | 03:46 | |
| docaedo | which for one thing is calling system PBR rather then venv (which makes me think it's not really staying in the venv) | 03:46 |
| mordred | hrm | 03:46 |
| mordred | docaedo: can you set cwd? | 03:47 |
| mordred | docaedo: to be /opt/apps_site | 03:48 |
| docaedo | yep, it's set to that | 03:48 |
| mordred | hrm | 03:48 |
| docaedo | http://paste.openstack.org/show/479202/ | 03:48 |
| mordred | it certainly does seem to be leaping to the system | 03:48 |
| *** achanda has quit IRC | 03:48 | |
| *** Sree has quit IRC | 03:49 | |
| *** Sree has joined #openstack-infra | 03:49 | |
| mordred | docaedo: can you conirm that /ops/app_site is a git repo? | 03:50 |
| mordred | also, can you confirm that git is installed? | 03:50 |
| mordred | OH | 03:50 |
| mordred | I got it | 03:50 |
| docaedo | mordred: for sure it is, vcsrepo is the one that gets it | 03:50 |
| mordred | path => '/opt/virtenv-app-catalog/bin', | 03:50 |
| mordred | docaedo: you don't have git in your path | 03:51 |
| *** weshay_xchat has quit IRC | 03:51 | |
| mordred | docaedo: because you have set path too restrictive | 03:51 |
| mordred | docaedo: you're gonna want /usr/bin in there at least | 03:51 |
| docaedo | mordred: ah ok - will add usr/bin and test | 03:51 |
| notmorgan | oh interesintg | 03:52 |
| *** jhesketh has quit IRC | 03:52 | |
| docaedo | holy moly that might have done it! | 03:52 |
| mordred | :) | 03:52 |
| * mordred drops git bombs | 03:52 | |
| docaedo | if it did, I'm taking the rest of the year off so I can end on a high note | 03:52 |
| *** calebb has joined #openstack-infra | 03:53 | |
| notmorgan | docaedo: at least the rest of november? ;) | 03:53 |
| *** jhesketh has joined #openstack-infra | 03:53 | |
| docaedo | haha | 03:53 |
| docaedo | thanks everyone, I think that was the last bit making this difficult, seems like everything works!! woooo! | 03:54 |
| mordred | yay! | 03:54 |
| docaedo | Have some cleanup work to do still but should be able to ask for infra to take a real look at it tomorrow - phew | 03:55 |
| *** mriedem has quit IRC | 03:55 | |
| *** rguillebert has quit IRC | 03:55 | |
| *** achanda has joined #openstack-infra | 03:55 | |
| docaedo | notmorgan: BTW welcome to the PNW where it might be dark and rainy but the beer is best in the country :) | 03:56 |
| *** eil397 has joined #openstack-infra | 03:56 | |
| notmorgan | docaedo: had some nice pizza today for dinner | 03:56 |
| notmorgan | docaedo: and a nice winter ale | 03:56 |
| nibalizer | haha why does that break it | 03:56 |
| nibalizer | oh because you dont' have git | 03:56 |
| nibalizer | because of the path | 03:56 |
| nibalizer | nice | 03:56 |
| notmorgan | nibalizer: fun, huh | 03:56 |
| openstackgerrit | Daniel DÃaz proposed openstack-infra/jenkins-job-builder: Add support for matrices' subset in multijobs. https://review.openstack.org/237082 | 03:56 |
| nibalizer | docaedo: left you some unrelated comments on your patch | 03:57 |
| docaedo | nibalizer: thank you! | 03:57 |
| clarkb | this is the best weather | 03:58 |
| clarkb | turn up the heat, drink stout, get drenched | 03:58 |
| notmorgan | clarkb: ++ | 04:00 |
| jroll | heh, ++ | 04:00 |
| notmorgan | clarkb: this weather is part of why I relocated here! :) | 04:00 |
| openstackgerrit | Christopher Aedo proposed openstack-infra/puppet-apps_site: Add python WSGI server https://review.openstack.org/245315 | 04:01 |
| notmorgan | clarkb: this weather > 80-85F winter in LA | 04:03 |
| *** Sree has quit IRC | 04:03 | |
| *** Sree has joined #openstack-infra | 04:04 | |
| *** amrith is now known as _amrith_ | 04:05 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add in extra connections tests https://review.openstack.org/169300 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Refactor sources out of triggers https://review.openstack.org/118993 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Move Item formatting into Reporters https://review.openstack.org/223298 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Document the new connections in zuul https://review.openstack.org/169270 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Remove ActionReporters https://review.openstack.org/243474 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add support for sqlalchemy reporter https://review.openstack.org/223333 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Tidy up loggers https://review.openstack.org/224336 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Configure triggers dynamically https://review.openstack.org/119534 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add support for 'connection' concept https://review.openstack.org/121528 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add base class for triggers https://review.openstack.org/119532 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add base class for sources https://review.openstack.org/119531 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add base class for reporters https://review.openstack.org/119530 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Tidy up tests/base.py https://review.openstack.org/238817 | 04:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add support to reject changes from approvals https://review.openstack.org/211108 | 04:06 |
| *** fawadkhaliq has quit IRC | 04:06 | |
| stevemar_ | since we're not supporting py26 for oslo stuff any longer, when is the massive patch coming for python client libs? | 04:06 |
| *** yuanying has joined #openstack-infra | 04:10 | |
| *** ddieterl_ has quit IRC | 04:11 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-infracloud: Add bifrost manifest https://review.openstack.org/231757 | 04:11 |
| *** erikmwilson_ has joined #openstack-infra | 04:15 | |
| *** erikwilson has quit IRC | 04:16 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Enhance CRD doc with diagrams https://review.openstack.org/229416 | 04:20 |
| *** fedexo has joined #openstack-infra | 04:22 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Add baremetal hpuswest node definition https://review.openstack.org/208751 | 04:22 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Add baremetal node definition for hpuseast https://review.openstack.org/246739 | 04:22 |
| openstackgerrit | fumihiko kakuma proposed openstack-infra/project-config: Add job for networking-ofagent https://review.openstack.org/244986 | 04:23 |
| *** fawadkhaliq has joined #openstack-infra | 04:27 | |
| *** amotoki has joined #openstack-infra | 04:34 | |
| *** Sree_ has joined #openstack-infra | 04:34 | |
| *** Sree_ is now known as Guest53303 | 04:35 | |
| *** eil397 has quit IRC | 04:35 | |
| openstackgerrit | Christopher Aedo proposed openstack-infra/puppet-apps_site: Add python WSGI server https://review.openstack.org/245315 | 04:35 |
| *** Guest53303 has quit IRC | 04:37 | |
| openstackgerrit | Merged openstack-infra/zuul: Add tests for --cache-dir option https://review.openstack.org/220615 | 04:37 |
| *** Sree has quit IRC | 04:37 | |
| *** Sree has joined #openstack-infra | 04:38 | |
| *** markvoelker_ has quit IRC | 04:40 | |
| openstackgerrit | Akihiro Motoki proposed openstack/requirements: Bump oslo.config >=2.7.0 https://review.openstack.org/246294 | 04:42 |
| *** bhunter71 has quit IRC | 04:43 | |
| *** annegentle has joined #openstack-infra | 04:47 | |
| *** otter768 has quit IRC | 04:47 | |
| *** _denisra has quit IRC | 04:49 | |
| *** amotoki has quit IRC | 04:49 | |
| *** annegentle has quit IRC | 04:52 | |
| *** jamielennox|away is now known as jamielennox | 04:54 | |
| *** paco20151113 has joined #openstack-infra | 04:55 | |
| *** Ramanjaneya has joined #openstack-infra | 04:57 | |
| *** yfried|afk is now known as yfried__ | 04:58 | |
| *** amotoki has joined #openstack-infra | 05:01 | |
| *** Sree has quit IRC | 05:02 | |
| *** Sree has joined #openstack-infra | 05:04 | |
| *** shashank_hegde has joined #openstack-infra | 05:04 | |
| *** akshai has quit IRC | 05:05 | |
| *** binbincong has joined #openstack-infra | 05:07 | |
| *** rossella_s has quit IRC | 05:07 | |
| *** bardia has joined #openstack-infra | 05:07 | |
| *** rossella_s has joined #openstack-infra | 05:08 | |
| bardia | asselin_: I have gerrit setup for my noop job. it triggers the job from patches and they pass but then i don't see any updates posted to the review page from the jenkins. | 05:09 |
| *** mtanino has quit IRC | 05:09 | |
| *** rick_ has quit IRC | 05:09 | |
| bardia | asselin_: is there anything I'm missing? shouldn't getting job triggers from patches indicate that it works? | 05:10 |
| *** amotoki has quit IRC | 05:10 | |
| asselin__ | bardia, need to double-check the project-config zuul layout.yaml file to make sure it's configured to post results | 05:10 |
| bardia | is it zuul that posts results or gerrit | 05:10 |
| *** sridhar_ram has joined #openstack-infra | 05:11 | |
| asselin__ | yes | 05:11 |
| bardia | shouldn't it be the gerrit plugin posting results after it got triggered? | 05:11 |
| asselin__ | there shouldn't be any gerrit plugin | 05:11 |
| *** tsg has joined #openstack-infra | 05:11 | |
| *** ddieterly has joined #openstack-infra | 05:11 | |
| asselin__ | did you install it manually? | 05:12 |
| *** achanda has quit IRC | 05:12 | |
| bardia | yeah | 05:14 |
| bardia | zuul was not working for me | 05:14 |
| *** amotoki has joined #openstack-infra | 05:14 | |
| asselin__ | what was not working? | 05:14 |
| bardia | the builds were not getting triggered after a patch | 05:15 |
| bardia | I guess I should check the zuul logs | 05:15 |
| asselin__ | I recommend you use zuul, but it's not required. | 05:16 |
| *** ddieterly has quit IRC | 05:17 | |
| *** Sukhdev has joined #openstack-infra | 05:17 | |
| asselin__ | but the docs and solution are assuming you're using that. | 05:17 |
| asselin__ | "not required" refers to 3rd party ci in general doesn't require you use to use any particular tool | 05:18 |
| *** jaypipes has quit IRC | 05:20 | |
| *** rick_ has joined #openstack-infra | 05:22 | |
| *** asselin__ has quit IRC | 05:22 | |
| *** asselin has joined #openstack-infra | 05:23 | |
| asselin | bardia, sorry, network failed | 05:23 |
| bardia | asselin: yeah I understand its not required but its the one thats supported and you are here to help :) | 05:24 |
| openstackgerrit | ChangBo Guo(gcb) proposed openstack/requirements: Bump oslo.utils to 2.8.0 https://review.openstack.org/246750 | 05:24 |
| asselin | bardia, yes that's the one I support along with the entire openstack infra team (ppl in this chat room) | 05:24 |
| greghaynes | jeblair: Yep, I think adding infra-cloud to nodepool as soon as we can would be awesome | 05:25 |
| bardia | asselin: I'm in the process of figuring out why zuul is not working. I'll let you know what I find | 05:25 |
| *** zhurong has quit IRC | 05:25 | |
| *** achanda has joined #openstack-infra | 05:25 | |
| greghaynes | jeblair: I think it isnt quite there, but it would also be great to get used to completing the loop on work | 05:25 |
| openstackgerrit | ChangBo Guo(gcb) proposed openstack/requirements: Bump oslo.utils to 2.8.0 https://review.openstack.org/246149 | 05:25 |
| *** cody-somerville has quit IRC | 05:27 | |
| *** stevemar_ has quit IRC | 05:27 | |
| *** stevemar_ has joined #openstack-infra | 05:28 | |
| *** zhurong has joined #openstack-infra | 05:29 | |
| *** krtaylor has quit IRC | 05:29 | |
| *** achanda has quit IRC | 05:29 | |
| sridhar_ram | infra team - recently (past week or so) there seems to be a gerrit issue where it no longer marking the bug status properly in the launchpad. For e.g. gerrit patchset with Closes-Bug is not marking the corresponding launchpad bug to In-Progress. In this happening for openstack/tacker project. LP: https://bugs.launchpad.net/tacker/+bug/1513591 and gerrit https://review.openstack.org/#/c/242248/ | 05:29 |
| openstack | Launchpad bug 1513591 in tacker "Testcase to automate http ping with respawn failure policy" [Low,Confirmed] - Assigned to Santosh (ksantosh-cs) | 05:29 |
| *** shashank_hegde has quit IRC | 05:30 | |
| sridhar_ram | any clue what could cause this ? | 05:30 |
| *** stevemar_ has quit IRC | 05:30 | |
| *** rpodolyaka1 has joined #openstack-infra | 05:33 | |
| *** achanda has joined #openstack-infra | 05:35 | |
| *** kushal has joined #openstack-infra | 05:37 | |
| *** rpodolyaka1 has quit IRC | 05:37 | |
| *** tsg has left #openstack-infra | 05:38 | |
| *** rpodolyaka1 has joined #openstack-infra | 05:38 | |
| *** amotoki has quit IRC | 05:40 | |
| *** cody-somerville has joined #openstack-infra | 05:43 | |
| *** rpodolyaka1 has quit IRC | 05:46 | |
| *** salv-orl_ has joined #openstack-infra | 05:50 | |
| *** Sukhdev has quit IRC | 05:50 | |
| *** rpodolyaka1 has joined #openstack-infra | 05:51 | |
| *** amotoki has joined #openstack-infra | 05:51 | |
| *** achanda has quit IRC | 05:53 | |
| *** salv-orlando has quit IRC | 05:53 | |
| *** gildub_ has quit IRC | 05:53 | |
| *** salv-orl_ has quit IRC | 05:55 | |
| jeblair | docaedo: do you _need_ to use a virtualenv? we generally do not use them on our servers. | 05:56 |
| jeblair | docaedo: there are other wsgi servers we run in infra, can you pattern your thing off of those? | 05:56 |
| asselin | jeblair, are there examples where a virtualenv are use? | 05:57 |
| jeblair | asselin: no, because there are none :) | 05:57 |
| docaedo | I do not need to, but it's working with it now? I can take a shot at making it work without if I need to :) | 05:57 |
| *** achanda has joined #openstack-infra | 05:58 | |
| asselin | jeblair, I'm stuck trying to resolve that urllib3 unsupported platform issue | 05:58 |
| jeblair | when i said 'generally' i meant 'in all cases' not 'in most cases' :) | 05:58 |
| jeblair | asselin: :( sorry, it's past my bedtime | 05:59 |
| jeblair | asselin: i can take a look at it with you tomorrow | 05:59 |
| *** stack_ has quit IRC | 06:01 | |
| *** nadya_ has joined #openstack-infra | 06:01 | |
| asselin | jeblair, sure thanks. I'm going to revert my attemps to fix it | 06:01 |
| docaedo | Ok, I'll take a shot at this with no venv tomorrow, can't wait to find new mysterious errors :) | 06:01 |
| *** stack_ has joined #openstack-infra | 06:02 | |
| openstackgerrit | daisy-ycguo proposed openstack-infra/project-config: Changed translation scripts to support i18n repo Add translation jobs for i18n repo https://review.openstack.org/245669 | 06:03 |
| *** gongysh__ has joined #openstack-infra | 06:06 | |
| *** rpodolyaka1 has quit IRC | 06:06 | |
| *** camunoz has joined #openstack-infra | 06:07 | |
| *** Liuqing has quit IRC | 06:10 | |
| *** Liuqing has joined #openstack-infra | 06:11 | |
| *** ddieterly has joined #openstack-infra | 06:13 | |
| *** gongysh__ has quit IRC | 06:15 | |
| *** hdd has joined #openstack-infra | 06:16 | |
| *** sridhar_ram has quit IRC | 06:16 | |
| *** sridhar_ram has joined #openstack-infra | 06:17 | |
| *** nadya_ has quit IRC | 06:18 | |
| *** harlowja_at_home has joined #openstack-infra | 06:18 | |
| *** ddieterly has quit IRC | 06:19 | |
| *** stack_ has quit IRC | 06:20 | |
| *** binbincong has quit IRC | 06:20 | |
| *** clayton has quit IRC | 06:22 | |
| *** changbl has quit IRC | 06:22 | |
| *** clayton has joined #openstack-infra | 06:23 | |
| *** changbl has joined #openstack-infra | 06:23 | |
| *** Sree_ has joined #openstack-infra | 06:24 | |
| *** Sree_ is now known as Guest40667 | 06:24 | |
| *** abregman has joined #openstack-infra | 06:25 | |
| *** rpodolyaka1 has joined #openstack-infra | 06:25 | |
| *** pcrews has quit IRC | 06:25 | |
| *** unicell has joined #openstack-infra | 06:26 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/246763 | 06:26 |
| *** sridhar_ram has left #openstack-infra | 06:27 | |
| *** Sree has quit IRC | 06:27 | |
| *** gnuoy has joined #openstack-infra | 06:27 | |
| *** jamespage has joined #openstack-infra | 06:27 | |
| *** rpodolyaka1 has quit IRC | 06:28 | |
| *** aeng has quit IRC | 06:29 | |
| *** rpodolyaka1 has joined #openstack-infra | 06:30 | |
| *** angdraug has joined #openstack-infra | 06:30 | |
| *** aeng has joined #openstack-infra | 06:31 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/246764 | 06:31 |
| *** nikil89_ has joined #openstack-infra | 06:34 | |
| *** harlowja_at_home has quit IRC | 06:35 | |
| openstackgerrit | Alexander Evseev proposed openstack-infra/zuul: Add possibility to use SSH instead of HTTP to get ref list https://review.openstack.org/246433 | 06:36 |
| *** rcernin has joined #openstack-infra | 06:37 | |
| openstackgerrit | Alexander Evseev proposed openstack-infra/zuul: Split pipeline description by double newlines on status page https://review.openstack.org/239943 | 06:39 |
| *** markvoelker has joined #openstack-infra | 06:40 | |
| *** binbincong has joined #openstack-infra | 06:43 | |
| *** markvoelker has quit IRC | 06:45 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add in extra connections tests https://review.openstack.org/169300 | 06:46 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Move Item formatting into Reporters https://review.openstack.org/223298 | 06:46 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Document the new connections in zuul https://review.openstack.org/169270 | 06:46 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Remove ActionReporters https://review.openstack.org/243474 | 06:46 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add support for sqlalchemy reporter https://review.openstack.org/223333 | 06:46 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Tidy up loggers https://review.openstack.org/224336 | 06:46 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add support for 'connection' concept https://review.openstack.org/121528 | 06:46 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Tidy up tests/base.py https://review.openstack.org/238817 | 06:46 |
| *** yfried__ has quit IRC | 06:47 | |
| *** sridhar_ram has joined #openstack-infra | 06:47 | |
| *** korzen has joined #openstack-infra | 06:48 | |
| *** aopincar has joined #openstack-infra | 06:48 | |
| *** annegentle has joined #openstack-infra | 06:48 | |
| *** otter768 has joined #openstack-infra | 06:48 | |
| *** salv-orlando has joined #openstack-infra | 06:51 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Publish i18n guides also from branches https://review.openstack.org/246284 | 06:52 |
| *** AJaeger has joined #openstack-infra | 06:52 | |
| *** otter768 has quit IRC | 06:53 | |
| AJaeger | ianw, yolanda : Could you review https://review.openstack.org/#/c/246284 today, please? It's a simple change to enable translations for manual jobs on branches. | 06:53 |
| *** annegentle has quit IRC | 06:53 | |
| *** MarkAtwood has joined #openstack-infra | 06:53 | |
| *** elemoine has joined #openstack-infra | 06:56 | |
| *** Ramanjaneya has quit IRC | 06:58 | |
| *** Ramanjaneya has joined #openstack-infra | 06:58 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/246763 | 06:58 |
| *** rpodolyaka1 has quit IRC | 06:59 | |
| *** rpodolyaka1 has joined #openstack-infra | 07:01 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/meetbot: Add warning if starting a meeting in an unrecognised channel https://review.openstack.org/246695 | 07:02 |
| *** angdraug has quit IRC | 07:03 | |
| *** gildub_ has joined #openstack-infra | 07:03 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/meetbot: Add warning if starting a meeting in an unrecognised channel https://review.openstack.org/246695 | 07:03 |
| *** Liuqing has quit IRC | 07:05 | |
| *** Liuqing has joined #openstack-infra | 07:06 | |
| *** tdurakov has joined #openstack-infra | 07:08 | |
| *** unicell1 has joined #openstack-infra | 07:08 | |
| *** unicell has quit IRC | 07:09 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/puppet-meetbot: Add config for recognised channels https://review.openstack.org/246778 | 07:10 |
| *** fedexo has quit IRC | 07:11 | |
| *** salv-orlando has quit IRC | 07:12 | |
| *** armax has quit IRC | 07:16 | |
| *** ddieterly has joined #openstack-infra | 07:16 | |
| *** ParsectiX has joined #openstack-infra | 07:17 | |
| *** gongysh__ has joined #openstack-infra | 07:17 | |
| *** tdurakov has quit IRC | 07:18 | |
| *** ddieterly has quit IRC | 07:20 | |
| *** tdurakov has joined #openstack-infra | 07:22 | |
| *** gildub_ has quit IRC | 07:24 | |
| *** Guest40667 has quit IRC | 07:24 | |
| *** Sree has joined #openstack-infra | 07:25 | |
| *** Sree has quit IRC | 07:29 | |
| *** tdurakov has quit IRC | 07:31 | |
| *** asselin__ has joined #openstack-infra | 07:32 | |
| *** zns has joined #openstack-infra | 07:32 | |
| *** achanda has quit IRC | 07:32 | |
| *** asselin_ has quit IRC | 07:33 | |
| *** korzen has quit IRC | 07:34 | |
| openstackgerrit | Sergey Nikitin proposed openstack/requirements: Updated oslo.db to 3.2.0 https://review.openstack.org/246325 | 07:36 |
| *** yfried__ has joined #openstack-infra | 07:38 | |
| *** dizquierdo has joined #openstack-infra | 07:39 | |
| *** serg_melikyan has quit IRC | 07:39 | |
| *** markvoelker has joined #openstack-infra | 07:41 | |
| *** gildub_ has joined #openstack-infra | 07:41 | |
| *** yfried has joined #openstack-infra | 07:41 | |
| *** AJaeger has quit IRC | 07:42 | |
| *** yfried__ has quit IRC | 07:42 | |
| *** markvoelker has quit IRC | 07:46 | |
| *** angdraug has joined #openstack-infra | 07:46 | |
| *** erikmwilson_ has quit IRC | 07:47 | |
| *** erikwilson has joined #openstack-infra | 07:48 | |
| *** sshnaidm has joined #openstack-infra | 07:49 | |
| *** scheuran has joined #openstack-infra | 07:49 | |
| *** yfried is now known as yfried|afk | 07:52 | |
| *** erikwilson has quit IRC | 07:52 | |
| *** bardia has quit IRC | 07:54 | |
| *** dizquierdo has quit IRC | 07:58 | |
| *** Ala has joined #openstack-infra | 07:58 | |
| *** sshnaidm has quit IRC | 08:03 | |
| *** yfried|afk is now known as yfried | 08:04 | |
| *** matrohon has joined #openstack-infra | 08:05 | |
| *** ifarkas has joined #openstack-infra | 08:08 | |
| *** Sree has joined #openstack-infra | 08:09 | |
| *** dtardivel has joined #openstack-infra | 08:11 | |
| *** stevemar_ has joined #openstack-infra | 08:12 | |
| *** MarkAtwood has quit IRC | 08:14 | |
| *** maishsk has joined #openstack-infra | 08:14 | |
| *** mkoderer has quit IRC | 08:15 | |
| *** stevemar_ has quit IRC | 08:16 | |
| *** ddieterly has joined #openstack-infra | 08:17 | |
| *** mkoderer has joined #openstack-infra | 08:17 | |
| *** sshnaidm has joined #openstack-infra | 08:17 | |
| *** salv-orlando has joined #openstack-infra | 08:19 | |
| *** shardy has joined #openstack-infra | 08:21 | |
| *** ddieterly has quit IRC | 08:21 | |
| *** Guest38114 has quit IRC | 08:22 | |
| *** dingyichen has quit IRC | 08:23 | |
| *** Liuqing has quit IRC | 08:23 | |
| *** Liuqing has joined #openstack-infra | 08:24 | |
| *** kozhukalov_ has joined #openstack-infra | 08:26 | |
| *** salv-orlando has quit IRC | 08:27 | |
| *** mahatic has joined #openstack-infra | 08:27 | |
| *** yfried_ has joined #openstack-infra | 08:27 | |
| *** yfried has quit IRC | 08:28 | |
| *** yamamoto has quit IRC | 08:28 | |
| *** egonzalez has joined #openstack-infra | 08:30 | |
| *** sridhar_ram has quit IRC | 08:30 | |
| *** k4n0 has joined #openstack-infra | 08:31 | |
| *** mpavone has joined #openstack-infra | 08:33 | |
| *** rick_ has quit IRC | 08:33 | |
| *** jcoufal has joined #openstack-infra | 08:35 | |
| openstackgerrit | daisy-ycguo proposed openstack-infra/project-config: Changed translation scripts and add translation jobs for I18n repo https://review.openstack.org/245669 | 08:36 |
| *** dtantsur|afk is now known as dtantsur | 08:36 | |
| *** arxcruz has joined #openstack-infra | 08:37 | |
| openstackgerrit | daisy-ycguo proposed openstack-infra/project-config: Change translation scripts and add translation jobs for I18n repo https://review.openstack.org/245669 | 08:37 |
| openstackgerrit | Anusha Ramineni proposed openstack-infra/project-config: Add specs-jobs to congress-specs https://review.openstack.org/246796 | 08:38 |
| *** fawadkhaliq has quit IRC | 08:41 | |
| *** hashar has joined #openstack-infra | 08:42 | |
| *** fawadkhaliq has joined #openstack-infra | 08:42 | |
| *** jlanoux has joined #openstack-infra | 08:45 | |
| *** camunoz has quit IRC | 08:46 | |
| *** annegentle has joined #openstack-infra | 08:49 | |
| *** jordanP has joined #openstack-infra | 08:49 | |
| *** otter768 has joined #openstack-infra | 08:49 | |
| *** fhubik has joined #openstack-infra | 08:50 | |
| *** kozhukalov_ has quit IRC | 08:51 | |
| *** Liuqing has quit IRC | 08:53 | |
| *** zeih has joined #openstack-infra | 08:53 | |
| *** annegentle has quit IRC | 08:53 | |
| *** luqas__ has joined #openstack-infra | 08:53 | |
| *** Liuqing has joined #openstack-infra | 08:54 | |
| *** otter768 has quit IRC | 08:54 | |
| *** zeih has quit IRC | 08:56 | |
| openstackgerrit | daisy-ycguo proposed openstack-infra/project-config: Add translation jobs and change scripts for I18n repo https://review.openstack.org/245669 | 08:56 |
| *** zhurong has quit IRC | 08:56 | |
| *** fhubik is now known as fhubik_brb | 08:56 | |
| *** fawadkhaliq has quit IRC | 08:57 | |
| *** zeih has joined #openstack-infra | 08:59 | |
| *** moravec has quit IRC | 09:00 | |
| *** ihrachys has joined #openstack-infra | 09:00 | |
| *** yfried__ has joined #openstack-infra | 09:01 | |
| *** fhubik_brb is now known as fhubik | 09:01 | |
| *** moravec has joined #openstack-infra | 09:01 | |
| *** yfried_ has quit IRC | 09:05 | |
| *** achanda has joined #openstack-infra | 09:05 | |
| *** mahatic has quit IRC | 09:05 | |
| *** rpodolyaka1 has quit IRC | 09:08 | |
| *** mahatic has joined #openstack-infra | 09:08 | |
| *** sshnaidm has quit IRC | 09:09 | |
| *** zeih has quit IRC | 09:12 | |
| *** stevemar_ has joined #openstack-infra | 09:13 | |
| *** zeih has joined #openstack-infra | 09:14 | |
| *** nadya_ has joined #openstack-infra | 09:15 | |
| *** ddieterly has joined #openstack-infra | 09:17 | |
| *** stevemar_ has quit IRC | 09:17 | |
| *** ParsectiX has quit IRC | 09:18 | |
| *** egonzalez has quit IRC | 09:20 | |
| *** Piet has quit IRC | 09:21 | |
| *** achanda has quit IRC | 09:22 | |
| *** ddieterly has quit IRC | 09:22 | |
| *** sshnaidm has joined #openstack-infra | 09:22 | |
| *** tdurakov_ has joined #openstack-infra | 09:24 | |
| *** salv-orlando has joined #openstack-infra | 09:26 | |
| *** yfried__ is now known as yfried|afk | 09:27 | |
| *** yfried|afk is now known as yfried__ | 09:27 | |
| *** ParsectiX has joined #openstack-infra | 09:28 | |
| *** fhubik is now known as fhubik_brb | 09:31 | |
| *** e0ne has joined #openstack-infra | 09:32 | |
| *** Liuqing has quit IRC | 09:32 | |
| *** zeih has quit IRC | 09:35 | |
| *** pblaho has joined #openstack-infra | 09:35 | |
| *** salv-orlando has quit IRC | 09:36 | |
| *** salv-orlando has joined #openstack-infra | 09:37 | |
| *** mpavone has quit IRC | 09:37 | |
| *** tdurakov_ has quit IRC | 09:37 | |
| *** Sree__ has joined #openstack-infra | 09:38 | |
| *** tdurakov_ has joined #openstack-infra | 09:38 | |
| *** msuriar_ is now known as msuriar | 09:38 | |
| *** zeih has joined #openstack-infra | 09:38 | |
| *** yamamoto has joined #openstack-infra | 09:39 | |
| *** tdurakov_ has quit IRC | 09:39 | |
| *** Hal has joined #openstack-infra | 09:41 | |
| *** Sree has quit IRC | 09:41 | |
| *** rpodolyaka1 has joined #openstack-infra | 09:41 | |
| *** Hal is now known as Guest88694 | 09:41 | |
| *** tdurakov_ has joined #openstack-infra | 09:42 | |
| *** markvoelker has joined #openstack-infra | 09:42 | |
| *** jistr has joined #openstack-infra | 09:42 | |
| *** yamamoto has quit IRC | 09:43 | |
| *** ZZelle has quit IRC | 09:44 | |
| *** ZZelle has joined #openstack-infra | 09:44 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/diskimage-builder: Add dynamic-login element https://review.openstack.org/245082 | 09:45 |
| *** markvoelker has quit IRC | 09:46 | |
| *** erikwilson has joined #openstack-infra | 09:48 | |
| *** fhubik_brb is now known as fhubik | 09:51 | |
| *** erikwilson has quit IRC | 09:53 | |
| openstackgerrit | Merged openstack-infra/project-config: Publish i18n guides also from branches https://review.openstack.org/246284 | 09:54 |
| *** rguillebert has joined #openstack-infra | 09:55 | |
| *** hichihara has quit IRC | 09:56 | |
| *** Yanyanhu has quit IRC | 09:57 | |
| *** ociuhandu has quit IRC | 10:00 | |
| *** paco20151113 has quit IRC | 10:00 | |
| *** Liuqing has joined #openstack-infra | 10:03 | |
| *** spredzy|afk is now known as spredzy | 10:04 | |
| *** zeih has quit IRC | 10:05 | |
| *** Sree__ has quit IRC | 10:10 | |
| *** Sree has joined #openstack-infra | 10:10 | |
| *** dzamboni has joined #openstack-infra | 10:18 | |
| *** ddieterly has joined #openstack-infra | 10:18 | |
| *** yfried__ is now known as yfried|afk | 10:19 | |
| *** Sree has quit IRC | 10:21 | |
| *** Sree has joined #openstack-infra | 10:22 | |
| *** ddieterly has quit IRC | 10:22 | |
| openstackgerrit | Merged openstack-infra/project-config: add python3 jobs to reno https://review.openstack.org/246579 | 10:23 |
| *** kozhukalov_ has joined #openstack-infra | 10:24 | |
| *** yamamoto has joined #openstack-infra | 10:26 | |
| *** Sree has quit IRC | 10:26 | |
| *** tdurakov_ has quit IRC | 10:27 | |
| *** daemontool has joined #openstack-infra | 10:28 | |
| openstackgerrit | Merged openstack-infra/puppet-infracloud: Add controller class and supporting manifests https://review.openstack.org/231631 | 10:28 |
| *** daemontool has quit IRC | 10:30 | |
| *** daemontool has joined #openstack-infra | 10:30 | |
| *** daemontool has quit IRC | 10:31 | |
| *** yamamoto has quit IRC | 10:31 | |
| openstackgerrit | Merged openstack-infra/puppet-infracloud: Add compute class https://review.openstack.org/231632 | 10:31 |
| *** ZZelle has quit IRC | 10:31 | |
| *** ZZelle has joined #openstack-infra | 10:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Don't run gate-horizon-dsvm-integration on stable branches https://review.openstack.org/246533 | 10:32 |
| igorbelikov | Hi folks, I see that gerrit downtime update email is not posted to openstack-dev ml (like the original announcement), only to openstack-infra. | 10:34 |
| igorbelikov | zaro anteaya nibalizer ^^ | 10:35 |
| openstackgerrit | Merged openstack-infra/project-config: add release notes jobs for ceilometer https://review.openstack.org/246458 | 10:35 |
| *** fawadkhaliq has joined #openstack-infra | 10:36 | |
| *** dtantsur is now known as dtantsur|brb | 10:37 | |
| *** ihrachys has quit IRC | 10:38 | |
| openstackgerrit | Merged openstack-infra/project-config: Install Heat for Sahara scenario jobs only if needed https://review.openstack.org/216605 | 10:38 |
| openstackgerrit | Merged openstack-infra/project-config: fix aodh gate https://review.openstack.org/246399 | 10:38 |
| *** ldnunes has joined #openstack-infra | 10:41 | |
| *** yfried|afk is now known as yfried__ | 10:42 | |
| *** nikil89_ is now known as naggappan | 10:43 | |
| *** e0ne has quit IRC | 10:43 | |
| *** dteselkin has quit IRC | 10:44 | |
| *** maiteb has joined #openstack-infra | 10:45 | |
| openstackgerrit | Igor Malinovskiy proposed openstack-infra/project-config: Use Manila specific job templates for building images https://review.openstack.org/233065 | 10:46 |
| *** amotoki has quit IRC | 10:47 | |
| *** dteselkin has joined #openstack-infra | 10:48 | |
| *** annegentle has joined #openstack-infra | 10:49 | |
| *** otter768 has joined #openstack-infra | 10:50 | |
| *** rpodolyaka1 has quit IRC | 10:52 | |
| *** fhubik is now known as fhubik_brb | 10:53 | |
| *** rpodolyaka1 has joined #openstack-infra | 10:53 | |
| *** annegentle has quit IRC | 10:54 | |
| *** otter768 has quit IRC | 10:54 | |
| *** aukhan has quit IRC | 10:56 | |
| *** sergmelikyan has joined #openstack-infra | 11:00 | |
| *** sergmelikyan has quit IRC | 11:02 | |
| *** aysyd has joined #openstack-infra | 11:03 | |
| *** e0ne has joined #openstack-infra | 11:04 | |
| *** jeblair has quit IRC | 11:04 | |
| *** ildikov has quit IRC | 11:05 | |
| *** jeblair has joined #openstack-infra | 11:05 | |
| *** jed56 has joined #openstack-infra | 11:06 | |
| *** sergmelikyan has joined #openstack-infra | 11:07 | |
| *** dims has joined #openstack-infra | 11:07 | |
| *** zhurong has joined #openstack-infra | 11:11 | |
| *** gongysh__ has quit IRC | 11:11 | |
| *** dims has quit IRC | 11:12 | |
| *** naggappan has quit IRC | 11:13 | |
| *** naggappan has joined #openstack-infra | 11:13 | |
| *** kushal has quit IRC | 11:14 | |
| *** stevemar_ has joined #openstack-infra | 11:14 | |
| *** fhubik_brb is now known as fhubik | 11:15 | |
| *** daemontool has joined #openstack-infra | 11:17 | |
| *** stevemar_ has quit IRC | 11:18 | |
| *** dims has joined #openstack-infra | 11:18 | |
| *** daemontool has quit IRC | 11:18 | |
| *** daemontool has joined #openstack-infra | 11:19 | |
| *** tdurakov_ has joined #openstack-infra | 11:20 | |
| *** daemontool has quit IRC | 11:21 | |
| *** tdurakov_ has quit IRC | 11:21 | |
| *** daemontool has joined #openstack-infra | 11:24 | |
| *** tdurakov_ has joined #openstack-infra | 11:24 | |
| *** sergmelikyan has quit IRC | 11:24 | |
| *** zhurong has quit IRC | 11:25 | |
| *** sergmelikyan has joined #openstack-infra | 11:25 | |
| *** fhubik is now known as fhubik_brb | 11:26 | |
| *** zhenguo has quit IRC | 11:27 | |
| *** sergmelikyan has quit IRC | 11:30 | |
| *** sergmelikyan has joined #openstack-infra | 11:31 | |
| *** yamamoto has joined #openstack-infra | 11:32 | |
| *** tdurakov_ has quit IRC | 11:32 | |
| *** dizquierdo has joined #openstack-infra | 11:33 | |
| *** mwagner has quit IRC | 11:34 | |
| *** tdurakov_ has joined #openstack-infra | 11:35 | |
| *** rfolco has joined #openstack-infra | 11:35 | |
| openstackgerrit | Maitê Balhester proposed openstack-infra/puppet-openstack_health: Fix httpd apply in Ubuntu Precise https://review.openstack.org/246053 | 11:36 |
| *** yamamoto has quit IRC | 11:36 | |
| *** ihrachys has joined #openstack-infra | 11:40 | |
| *** markvoelker has joined #openstack-infra | 11:43 | |
| *** dtantsur|brb is now known as dtantsur | 11:43 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: min_compute_nodes set before NOTESTS check https://review.openstack.org/246452 | 11:43 |
| *** claudiub has joined #openstack-infra | 11:44 | |
| *** daemontool has quit IRC | 11:47 | |
| *** markvoelker has quit IRC | 11:47 | |
| *** daemontool has joined #openstack-infra | 11:47 | |
| *** amotoki has joined #openstack-infra | 11:48 | |
| *** amotoki has quit IRC | 11:49 | |
| *** erikwilson has joined #openstack-infra | 11:49 | |
| *** fhubik_brb is now known as fhubik | 11:49 | |
| *** erikwilson has quit IRC | 11:54 | |
| *** ildikov has joined #openstack-infra | 11:54 | |
| *** jhesketh has quit IRC | 11:55 | |
| *** daniloramalho has joined #openstack-infra | 11:56 | |
| *** gildub_ has quit IRC | 11:59 | |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Rework ec2api jobs for new tempest`s plugin technology https://review.openstack.org/246890 | 12:00 |
| *** Sree has joined #openstack-infra | 12:00 | |
| *** jhesketh has joined #openstack-infra | 12:00 | |
| *** doug-fish has joined #openstack-infra | 12:02 | |
| *** pedroalvarez has joined #openstack-infra | 12:02 | |
| *** egonzalez has joined #openstack-infra | 12:03 | |
| *** gildub_ has joined #openstack-infra | 12:03 | |
| pedroalvarez | Hi! I'm trying to test that openstackid.org works fine with storyboard using the URL https://openstackid.org/accounts/openid2. Looks like it does, but it's failing because is not giving access to the name (and I guess nothing else). | 12:10 |
| *** rpodolyaka1 has quit IRC | 12:12 | |
| *** mission712 has quit IRC | 12:13 | |
| *** tdurakov_ has quit IRC | 12:14 | |
| *** placko_ has joined #openstack-infra | 12:15 | |
| *** dramalho has joined #openstack-infra | 12:15 | |
| *** tdurakov_ has joined #openstack-infra | 12:16 | |
| *** gildub_ has quit IRC | 12:17 | |
| *** ociuhandu has joined #openstack-infra | 12:19 | |
| *** ddieterly has joined #openstack-infra | 12:19 | |
| *** dramalho has quit IRC | 12:20 | |
| *** wolsen has quit IRC | 12:20 | |
| *** ociuhandu_ has joined #openstack-infra | 12:22 | |
| *** egonzalez has quit IRC | 12:22 | |
| *** ociuhandu has quit IRC | 12:23 | |
| *** ociuhandu_ is now known as ociuhandu | 12:23 | |
| *** ddieterly has quit IRC | 12:24 | |
| *** ddieterly has joined #openstack-infra | 12:24 | |
| *** aarefiev has quit IRC | 12:25 | |
| *** aarefiev has joined #openstack-infra | 12:27 | |
| *** rhallisey has joined #openstack-infra | 12:28 | |
| *** kushal has joined #openstack-infra | 12:28 | |
| *** jkraj has joined #openstack-infra | 12:28 | |
| *** thorst has joined #openstack-infra | 12:28 | |
| *** daniloramalho has left #openstack-infra | 12:32 | |
| *** daniloramalho has joined #openstack-infra | 12:32 | |
| *** ihrachys_ has joined #openstack-infra | 12:34 | |
| *** ihrachys has quit IRC | 12:35 | |
| daniloramalho | Hello nibalizer crinkle Can you look to this patch https://review.openstack.org/#/c/246053/, please? | 12:35 |
| *** fhubik is now known as fhubik_brb | 12:37 | |
| *** yamamoto has joined #openstack-infra | 12:39 | |
| *** fawadkhaliq has quit IRC | 12:42 | |
| *** zhurong has joined #openstack-infra | 12:43 | |
| *** markvoelker has joined #openstack-infra | 12:43 | |
| openstackgerrit | Dmitry Tantsur proposed openstack-infra/project-config: Use prebuilt IPA image for ironic-inspector IPA gate job https://review.openstack.org/246907 | 12:43 |
| *** hdd has quit IRC | 12:44 | |
| *** gongysh__ has joined #openstack-infra | 12:44 | |
| *** fhubik_brb is now known as fhubik | 12:45 | |
| *** hdd has joined #openstack-infra | 12:46 | |
| *** markvoelker has quit IRC | 12:48 | |
| *** annegentle has joined #openstack-infra | 12:50 | |
| *** otter768 has joined #openstack-infra | 12:51 | |
| openstackgerrit | Ryan Hallisey proposed openstack-infra/project-config: Add a job around using tripleo with docker https://review.openstack.org/238475 | 12:52 |
| *** naggappan has quit IRC | 12:53 | |
| *** garyk1 has joined #openstack-infra | 12:53 | |
| *** otter768 has quit IRC | 12:55 | |
| *** annegentle has quit IRC | 12:55 | |
| *** tdurakov_ has quit IRC | 12:57 | |
| *** electrofelix has joined #openstack-infra | 12:57 | |
| *** e0ne has quit IRC | 12:57 | |
| *** zhurong has quit IRC | 12:57 | |
| *** placko__ has joined #openstack-infra | 12:57 | |
| *** mpavone has joined #openstack-infra | 12:57 | |
| *** zhurong has joined #openstack-infra | 12:58 | |
| *** tdurakov_ has joined #openstack-infra | 12:58 | |
| *** EricGonczer_ has joined #openstack-infra | 12:58 | |
| *** hdd has quit IRC | 12:59 | |
| *** placko_ has quit IRC | 13:00 | |
| *** weshay_xchat has joined #openstack-infra | 13:01 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Add support for matrices' subset in multijobs. https://review.openstack.org/237082 | 13:02 |
| openstackgerrit | Li Yingjun proposed openstack-infra/project-config: Add python-searchlightclient repo https://review.openstack.org/246691 | 13:04 |
| *** jyuso1 has quit IRC | 13:04 | |
| *** angdraug has quit IRC | 13:05 | |
| openstackgerrit | Anastasia Kuznetsova proposed openstack-infra/project-config: Update jobs for mistral https://review.openstack.org/246917 | 13:06 |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Rework ec2api jobs for new tempest`s plugin technology https://review.openstack.org/246890 | 13:07 |
| *** rossella_s has quit IRC | 13:07 | |
| *** gordc has joined #openstack-infra | 13:07 | |
| *** moravec has quit IRC | 13:08 | |
| *** rossella_s has joined #openstack-infra | 13:08 | |
| *** hdd has joined #openstack-infra | 13:09 | |
| *** markvoelker has joined #openstack-infra | 13:09 | |
| *** gordc_ has joined #openstack-infra | 13:09 | |
| *** e0ne has joined #openstack-infra | 13:10 | |
| *** Ramanjaneya has quit IRC | 13:12 | |
| *** gordc has quit IRC | 13:13 | |
| *** aopincar has quit IRC | 13:14 | |
| *** stevemar_ has joined #openstack-infra | 13:15 | |
| *** peristeri has joined #openstack-infra | 13:17 | |
| *** stevemar_ has quit IRC | 13:18 | |
| *** ihrachys_ has quit IRC | 13:18 | |
| *** ihrachys has joined #openstack-infra | 13:19 | |
| *** dprince has joined #openstack-infra | 13:19 | |
| *** angdraug has joined #openstack-infra | 13:21 | |
| *** EricGonc_ has joined #openstack-infra | 13:22 | |
| *** EricGonczer_ has quit IRC | 13:25 | |
| *** rpodolyaka1 has joined #openstack-infra | 13:26 | |
| *** annegentle has joined #openstack-infra | 13:26 | |
| *** hdd has quit IRC | 13:28 | |
| *** aopincar has joined #openstack-infra | 13:29 | |
| *** hdd has joined #openstack-infra | 13:29 | |
| *** Liuqing_ has joined #openstack-infra | 13:31 | |
| *** Liuqing has quit IRC | 13:33 | |
| *** skraynev has quit IRC | 13:34 | |
| *** gordc_ has quit IRC | 13:34 | |
| *** maiteb has quit IRC | 13:40 | |
| *** aopincar has quit IRC | 13:41 | |
| *** mwagner has joined #openstack-infra | 13:43 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: telemetry: don't share queue between projects https://review.openstack.org/241205 | 13:43 |
| *** achanda has joined #openstack-infra | 13:44 | |
| *** skraynev has joined #openstack-infra | 13:44 | |
| *** bhunter71 has joined #openstack-infra | 13:44 | |
| *** krtaylor has joined #openstack-infra | 13:45 | |
| *** tlian has joined #openstack-infra | 13:45 | |
| *** achanda has quit IRC | 13:46 | |
| *** amuller has joined #openstack-infra | 13:47 | |
| mordred | infra-root: I've created an internap account. I am waiting on internap to call me for verification purposes | 13:47 |
| *** achanda has joined #openstack-infra | 13:49 | |
| *** erikwilson has joined #openstack-infra | 13:50 | |
| openstackgerrit | Merged openstack-infra/system-config: Parameterize setting the selinux mode https://review.openstack.org/214737 | 13:52 |
| *** thiagop has joined #openstack-infra | 13:52 | |
| *** bcourt has quit IRC | 13:52 | |
| *** tiswanso has quit IRC | 13:53 | |
| *** achanda has quit IRC | 13:53 | |
| *** eharney has joined #openstack-infra | 13:53 | |
| daniloramalho | Hello mordred: Can you look to this patch https://review.openstack.org/#/c/246053/, please? | 13:54 |
| *** ddieterly has quit IRC | 13:54 | |
| *** erikwilson has quit IRC | 13:54 | |
| *** placko__ has quit IRC | 13:54 | |
| *** julim has joined #openstack-infra | 13:55 | |
| *** elemoine has quit IRC | 13:56 | |
| *** dizquierdo has quit IRC | 13:57 | |
| *** bdemers has joined #openstack-infra | 13:57 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Create ansible-role-jobs for JJB and zuul https://review.openstack.org/246633 | 13:57 |
| mordred | daniloramalho: done | 13:58 |
| *** doug-fish has quit IRC | 13:58 | |
| daniloramalho | mordred: Thanks :) | 13:58 |
| *** hdd has quit IRC | 13:59 | |
| *** MarkAtwood has joined #openstack-infra | 13:59 | |
| *** tdurakov_ has quit IRC | 13:59 | |
| mordred | clarkb: is 212745 still required? I believe we did something with groovy right? | 14:00 |
| mordred | clarkb: it's got a second +2 on it now, so if it is, feel free to approve | 14:01 |
| *** changbl has quit IRC | 14:01 | |
| *** bcourt has joined #openstack-infra | 14:01 | |
| *** burgerk has joined #openstack-infra | 14:02 | |
| *** maiteb has joined #openstack-infra | 14:04 | |
| *** akshai has joined #openstack-infra | 14:04 | |
| *** erikwilson has joined #openstack-infra | 14:04 | |
| openstackgerrit | Merged openstack-infra/infra-manual: Edit instructions for repo creation, include Needed-By https://review.openstack.org/221433 | 14:04 |
| *** erikwilson has quit IRC | 14:05 | |
| *** lykinsbd has joined #openstack-infra | 14:05 | |
| *** jffische has joined #openstack-infra | 14:06 | |
| *** tdurakov_ has joined #openstack-infra | 14:06 | |
| *** zhurong has quit IRC | 14:07 | |
| *** placko__ has joined #openstack-infra | 14:07 | |
| *** kozhukalov_ has quit IRC | 14:07 | |
| *** zhurong has joined #openstack-infra | 14:07 | |
| *** jffische has quit IRC | 14:08 | |
| *** bapalm has joined #openstack-infra | 14:08 | |
| openstackgerrit | Merged openstack-infra/nodepool: Add 'debian' user in bootstrapServer() https://review.openstack.org/221873 | 14:08 |
| *** boris-42 has quit IRC | 14:08 | |
| *** vlaza has joined #openstack-infra | 14:09 | |
| *** hdd has joined #openstack-infra | 14:09 | |
| *** fhubik is now known as fhubik_brb | 14:09 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Example of how to use targets plugins https://review.openstack.org/243865 | 14:09 |
| *** bswartz has joined #openstack-infra | 14:09 | |
| *** sergmelikyan has quit IRC | 14:09 | |
| openstackgerrit | Merged openstack-infra/system-config: Update the sample puppet code block for etherpad https://review.openstack.org/239723 | 14:10 |
| *** rlandy has joined #openstack-infra | 14:11 | |
| openstackgerrit | Merged openstack-infra/puppet-openstack_health: Fix httpd apply in Ubuntu Precise https://review.openstack.org/246053 | 14:11 |
| *** burgerk has quit IRC | 14:11 | |
| mordred | jesusaurus: 194305 there is a question about motivation in the review from clarkb | 14:11 |
| *** gongysh__ has quit IRC | 14:12 | |
| *** angdraug has quit IRC | 14:12 | |
| *** achanda has joined #openstack-infra | 14:12 | |
| *** zz_dimtruck is now known as dimtruck | 14:13 | |
| mordred | russellb: 174952 has a not-a-minus-one from you, but you said you were thinking about it - I'd rather not create a repo and then destroy it later - any final pushback? | 14:13 |
| *** MarkAtwood has quit IRC | 14:13 | |
| *** zhurong has quit IRC | 14:15 | |
| *** shashank_hegde has joined #openstack-infra | 14:16 | |
| mordred | jeblair: gertty feature request: it would be cool if when gertty would show me the aggregate merged status of depends-on patches in the UI | 14:18 |
| *** yamamoto has quit IRC | 14:19 | |
| *** kozhukalov_ has joined #openstack-infra | 14:19 | |
| hashar | mordred: thanks for my Nodepool patches A+1 :-} | 14:20 |
| *** fhubik_brb is now known as fhubik | 14:20 | |
| hashar | ""I'm actually looking forward to the day when we're not using provider provided images."" <--- I am using provider images myself since disk image builder requires root sudo on the Nodepool host :D | 14:20 |
| hashar | and root is a no/no for us | 14:20 |
| *** doug-fish has joined #openstack-infra | 14:21 | |
| mordred | hashar: is root sudo a no/no for you everywhere, or just on that host? (we have image-build-workers coming) | 14:21 |
| openstackgerrit | Merged openstack-infra/project-config: Added new repo for fuel-plugin-plumgrid in openstack NS https://review.openstack.org/240258 | 14:21 |
| hashar | mordred: in production realm it is a no/no. But we might have workers on some staging / labs area :-} | 14:22 |
| hashar | seems that image-build-workers is similar to the zuul merge operations being decentralized | 14:22 |
| *** gongysh__ has joined #openstack-infra | 14:22 | |
| *** jaypipes has joined #openstack-infra | 14:23 | |
| hashar | anyway my base image on the provider is very basic. I then rely on puppet and setup.sh to bring it up to an acceptable level and refresh a new snapshot | 14:23 |
| hashar | so not much uses about disk image :-} | 14:23 |
| openstackgerrit | Merged openstack-infra/puppet-log_processor: Create subunit proccessor subclass https://review.openstack.org/190744 | 14:23 |
| mordred | hashar: k. well, it's good to konw that keeping the snapshot image builder is important to you | 14:24 |
| *** gongysh__ has quit IRC | 14:25 | |
| *** daemontool_ has joined #openstack-infra | 14:25 | |
| *** gongysh__ has joined #openstack-infra | 14:26 | |
| *** daemontool_ has quit IRC | 14:26 | |
| *** ddieterly has joined #openstack-infra | 14:26 | |
| hashar | mordred: took me a while to figure out the difference between provider images and the refresh/snapshot system | 14:26 |
| *** ddieterly has quit IRC | 14:26 | |
| *** shashank_hegde has quit IRC | 14:26 | |
| hashar | might be worth asking other third parties and see how they use nodepool | 14:26 |
| *** daemontool_ has joined #openstack-infra | 14:27 | |
| *** Sree has quit IRC | 14:27 | |
| *** ddieterly has joined #openstack-infra | 14:27 | |
| *** rossella_s has quit IRC | 14:27 | |
| hashar | I would not be surprised if most of them use an outdated base image and rely on setup.sh to properly configure the snapshot | 14:27 |
| *** rossella_s has joined #openstack-infra | 14:27 | |
| *** daemontool has quit IRC | 14:27 | |
| *** kgiusti has joined #openstack-infra | 14:27 | |
| *** rossella_s has quit IRC | 14:27 | |
| *** Sree has joined #openstack-infra | 14:27 | |
| *** angdraug has joined #openstack-infra | 14:27 | |
| *** sergmelikyan has joined #openstack-infra | 14:28 | |
| *** rossella_s has joined #openstack-infra | 14:28 | |
| mordred | yah, I mean, we still use that for half of our images | 14:28 |
| mordred | as we haven't fully finished making the transition yet | 14:28 |
| *** tiswanso has joined #openstack-infra | 14:28 | |
| *** daemontool_ has quit IRC | 14:28 | |
| *** daemontool_ has joined #openstack-infra | 14:29 | |
| *** rossella_s has quit IRC | 14:29 | |
| *** rossella_s has joined #openstack-infra | 14:29 | |
| *** smarcet has joined #openstack-infra | 14:29 | |
| smarcet | morning | 14:31 |
| smarcet | https://login.launchpad.net/+openid is timeouting | 14:31 |
| *** openstackgerrit has quit IRC | 14:31 | |
| *** Sree has quit IRC | 14:32 | |
| *** openstackgerrit has joined #openstack-infra | 14:32 | |
| mordred | yay! | 14:32 |
| mordred | smarcet: just worked for me | 14:33 |
| smarcet | ok looking | 14:33 |
| smarcet | thx | 14:33 |
| sc68cal | worked for me | 14:33 |
| *** aspiers has quit IRC | 14:34 | |
| *** rkukura has quit IRC | 14:35 | |
| *** abregman has quit IRC | 14:35 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/irc-meetings: Added weekly meeting for Neutron Upgrades subteam https://review.openstack.org/246949 | 14:35 |
| *** abregman has joined #openstack-infra | 14:35 | |
| *** jkraj has quit IRC | 14:37 | |
| *** Swami has joined #openstack-infra | 14:39 | |
| *** _amrith_ is now known as amrith | 14:41 | |
| clarkb | pedroalvarez: smarcet eould be a good person to ask your openid question | 14:41 |
| clarkb | pedroalvarez: I think I am seeing similar with mod auth openid | 14:41 |
| *** bapalm has quit IRC | 14:43 | |
| *** angdraug has quit IRC | 14:43 | |
| pedroalvarez | I guess it's either menot configuring it properly, either storyboard needs some sort of new support for openstackid.org | 14:43 |
| *** angdraug has joined #openstack-infra | 14:43 | |
| *** bapalm has joined #openstack-infra | 14:43 | |
| clarkb | smarcet should be able to help | 14:44 |
| *** ddieterly has quit IRC | 14:44 | |
| clarkb | if you can relay the details here | 14:44 |
| *** fawadkhaliq has joined #openstack-infra | 14:44 | |
| *** dizquierdo has joined #openstack-infra | 14:45 | |
| *** mriedem has joined #openstack-infra | 14:45 | |
| *** mkwiek07 has joined #openstack-infra | 14:48 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Add user group assignment API https://review.openstack.org/246963 | 14:48 |
| Shrews | mordred: ^^^ | 14:48 |
| mkwiek07 | hello, can anyone help me with figuring out what's wrong with gate-fuel-astute-rake-dsvm check on this review: https://review.openstack.org/#/c/245769/ | 14:50 |
| *** vlaza has left #openstack-infra | 14:50 | |
| *** kozhukalov_ has quit IRC | 14:51 | |
| *** daemontool_ has quit IRC | 14:51 | |
| *** otter768 has joined #openstack-infra | 14:52 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:52 | |
| *** garyk1 has quit IRC | 14:52 | |
| phschwartz | Morning | 14:54 |
| mkwiek07 | this is link that jenkins provided: http://logs.openstack.org/69/245769/6/check/gate-fuel-astute-rake-dsvm/9cc491f/ unfortunately, it's not very informative :( | 14:54 |
| daniloramalho | phschwartz: Morning | 14:54 |
| *** Swami has quit IRC | 14:55 | |
| openstackgerrit | Claudiu Belu proposed openstack-infra/project-config: Enable Python 3 unit tests for os-win https://review.openstack.org/246969 | 14:55 |
| pabelanger | mkwiek07: I mentioned this yesterday to igorbelikov but you should be using gate-{name}-puppet-unit-{puppet_version}-dsvm-{ostype}. Your job definition is currently broken because you are missing ostype in the name. To launch devstack-centos nodes, you need dsvm-centos7 in the job name. It was also decided to make gate-{name}-puppet-unit-{puppet_version}-dsvm-{ostype} more generic. | 14:56 |
| *** otter768 has quit IRC | 14:56 | |
| pabelanger | mkwiek07: so, rather then use a duplicate job, switch it to the puppet job, then rename it to something more generic | 14:57 |
| *** ddieterly has joined #openstack-infra | 14:57 | |
| mkwiek07 | pabelanger: thank you! | 14:58 |
| *** Swami has joined #openstack-infra | 14:59 | |
| *** abregman has quit IRC | 15:00 | |
| igorbelikov | pabelanger: thanks for the ostype info, was not expecting current definition to be broken:( | 15:00 |
| *** ddieterly has quit IRC | 15:01 | |
| *** daemontool has joined #openstack-infra | 15:01 | |
| *** ddieterly has joined #openstack-infra | 15:01 | |
| *** dramalho has joined #openstack-infra | 15:01 | |
| *** hdd has quit IRC | 15:02 | |
| pabelanger | igorbelikov: Ya, we need to do a better job at catching jobs with dsvm in the name. There is some logic in zuul/openstack_functions.py to setup zuul node params. | 15:02 |
| *** kozhukalov_ has joined #openstack-infra | 15:03 | |
| *** angdraug has quit IRC | 15:03 | |
| EmilienM | hello, I have a quick system-config patch about IRC channel, if anyone can look : https://review.openstack.org/#/c/242621/ - thanks! | 15:03 |
| openstackgerrit | Merged openstack-infra/shade: Stop using uuid in functional tests https://review.openstack.org/245817 | 15:04 |
| openstackgerrit | Merged openstack-infra/shade: Add test for os_nova_flavor Ansible module https://review.openstack.org/245830 | 15:04 |
| *** signed8bit has joined #openstack-infra | 15:04 | |
| *** ParsectiX has quit IRC | 15:04 | |
| *** tiswanso has quit IRC | 15:06 | |
| *** dramalho has quit IRC | 15:06 | |
| mordred | Shrews: oh wow. that keystone api bit is awesome | 15:07 |
| Shrews | mordred: isn't it though? | 15:08 |
| *** alivigni has joined #openstack-infra | 15:08 | |
| mordred | Shrews: so, I had this thought about our use of random ids in our functional tests and a future state where we might use betamax against live clouds and then replay that | 15:09 |
| mordred | Shrews: and random names don't work nearly as well with replaying http responses from clouds | 15:09 |
| mordred | Shrews: made me think - why not base the names off of the method name of the currently running test? | 15:10 |
| mordred | that way its still unique, but it's also consistent from run to run which would make doing an http requestts replay more easier | 15:10 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/project-config: Only run gate-tempest-dsvm-multinode-live-migration on master https://review.openstack.org/246977 | 15:11 |
| *** Liuqing_ has quit IRC | 15:11 | |
| *** sabeen1 has joined #openstack-infra | 15:11 | |
| mordred | \o/ | 15:12 |
| mordred | internap has determined that we are not fraudulent | 15:12 |
| *** sabeen2 has joined #openstack-infra | 15:12 | |
| yolanda | :) | 15:12 |
| *** sergmelikyan has quit IRC | 15:12 | |
| Shrews | mordred: that's how getUniqueString() works, except it tacks on a unique integer to the test name | 15:14 |
| mordred | Shrews: also, that throws a keystoneauth1 exception? OH - I guess becaue that's a ... wow | 15:14 |
| mordred | Shrews: oh - well that's cool then | 15:14 |
| Shrews | mordred: https://github.com/testing-cabal/testtools/blob/master/testtools/testcase.py#L545-L547 | 15:14 |
| *** gongysh__ has quit IRC | 15:14 | |
| mordred | I wonder if we could get it to not tack on the unique integer | 15:14 |
| Shrews | mordred: so we *could* use self.id, maybe? | 15:15 |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr: Fail if no tests were successful executed https://review.openstack.org/246980 | 15:15 |
| mordred | well - but then we have the problem of like where you need a user and a group | 15:15 |
| *** sshnaidm has quit IRC | 15:16 | |
| *** stevemar_ has joined #openstack-infra | 15:16 | |
| mordred | Shrews: it seems like getUniqueString('user') should return self.id() + 'user' | 15:16 |
| *** cloudtrainme has joined #openstack-infra | 15:16 | |
| *** larainema has quit IRC | 15:16 | |
| *** sabeen1 has quit IRC | 15:16 | |
| *** whoops has quit IRC | 15:16 | |
| mordred | since that's guaranteed to be unique and also allows specifying which string you wnat for this test | 15:16 |
| *** whoops has joined #openstack-infra | 15:16 | |
| Shrews | mordred: no, it's actually 'user' + unique-int | 15:16 |
| *** niska` has quit IRC | 15:16 | |
| * mordred will bug lifeless whence he awakens | 15:16 | |
| *** jcoufal has quit IRC | 15:17 | |
| mordred | right | 15:17 |
| mordred | Shrews: I'm saying I'd like it to behave differently | 15:17 |
| *** sballe has quit IRC | 15:17 | |
| *** d34dh0r53 has quit IRC | 15:17 | |
| *** angdraug has joined #openstack-infra | 15:17 | |
| Shrews | mordred: https://github.com/testing-cabal/testtools/blob/master/testtools/testcase.py#L762-L763 | 15:17 |
| Shrews | just use that? | 15:17 |
| *** Nakato has quit IRC | 15:17 | |
| *** d34dh0r53 has joined #openstack-infra | 15:17 | |
| *** rbrndt has joined #openstack-infra | 15:17 | |
| Shrews | i might be missing your argument | 15:18 |
| *** larainema has joined #openstack-infra | 15:18 | |
| mordred | I might not be making it well | 15:18 |
| *** sballe has joined #openstack-infra | 15:18 | |
| *** yfried has joined #openstack-infra | 15:18 | |
| mordred | Shrews: I'll try again when I have second coffee | 15:18 |
| *** Nakato has joined #openstack-infra | 15:18 | |
| * mriedem is very happy to see e-r bot working again | 15:18 | |
| Shrews | mordred: oh, you mean the case where we need multiple unique strings in a single test | 15:18 |
| *** stevemar_ has quit IRC | 15:18 | |
| *** yfried__ has quit IRC | 15:19 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Add release notes jobs for ironic https://review.openstack.org/242152 | 15:19 |
| *** niska has joined #openstack-infra | 15:19 | |
| *** Steap_ has quit IRC | 15:19 | |
| *** yamamoto has joined #openstack-infra | 15:20 | |
| *** sdake has quit IRC | 15:20 | |
| Shrews | mordred: yeah, not sure why the keystoneauth1 exception is being thrown there... seems like an unintended feature to me | 15:20 |
| *** sshnaidm has joined #openstack-infra | 15:20 | |
| *** sdake has joined #openstack-infra | 15:20 | |
| *** Steap has joined #openstack-infra | 15:20 | |
| *** rpodolyaka1 has quit IRC | 15:20 | |
| *** tonytan4ever has joined #openstack-infra | 15:21 | |
| *** amrith is now known as _amrith_ | 15:21 | |
| *** rpodolyaka1 has joined #openstack-infra | 15:22 | |
| mordred | Shrews: I believe it's because it's the keystoneauth1 Session that's doing the requests calls, which means if it gets a 404 I guess it's throwing the NotFound | 15:22 |
| *** rlandy has quit IRC | 15:23 | |
| *** jcoufal has joined #openstack-infra | 15:23 | |
| *** cbader has joined #openstack-infra | 15:24 | |
| *** yamamoto has quit IRC | 15:25 | |
| *** dkranz has joined #openstack-infra | 15:27 | |
| *** pradk has joined #openstack-infra | 15:28 | |
| *** tiswanso has joined #openstack-infra | 15:28 | |
| *** ccrouch has joined #openstack-infra | 15:28 | |
| *** ildikov has quit IRC | 15:28 | |
| *** yonglihe has joined #openstack-infra | 15:29 | |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr: Fail if no tests were successful executed https://review.openstack.org/246980 | 15:29 |
| *** _amrith_ is now known as amrith | 15:30 | |
| *** salv-orlando has quit IRC | 15:30 | |
| *** ifarkas has quit IRC | 15:30 | |
| *** ifarkas has joined #openstack-infra | 15:30 | |
| *** ifarkas has quit IRC | 15:30 | |
| *** ccrouch has quit IRC | 15:30 | |
| *** salv-orlando has joined #openstack-infra | 15:31 | |
| *** ifarkas has joined #openstack-infra | 15:31 | |
| *** ccrouch has joined #openstack-infra | 15:31 | |
| *** duartea has quit IRC | 15:34 | |
| *** amrith is now known as _amrith_ | 15:38 | |
| openstackgerrit | Igor Belikov proposed openstack-infra/project-config: Switch fuel-astute and fuel-nailgun-agent gates to use puppet templates https://review.openstack.org/246989 | 15:38 |
| openstackgerrit | Igor Belikov proposed openstack-infra/project-config: Switch fuel-astute and fuel-nailgun-agent gates to use puppet templates https://review.openstack.org/246989 | 15:39 |
| *** k4n0 has quit IRC | 15:39 | |
| *** kozhukalov_ has quit IRC | 15:40 | |
| igorbelikov | pabelanger: please review ^^ | 15:40 |
| *** stevemar_ has joined #openstack-infra | 15:40 | |
| *** _amrith_ is now known as amrith | 15:41 | |
| *** rlandy has joined #openstack-infra | 15:41 | |
| igorbelikov | making puppet jobs more generic requires a bit deeper dive, can't do it right now, but it's on my todo list for this week:) | 15:42 |
| samueldmq | if I were to create a gate job for keystone with a specific configuration, such as federation, using ldap, or whatever | 15:42 |
| samueldmq | what would be the right approach to it? create a gate job in project-config with a specialized script to do the setup of ldap,idp,etc | 15:43 |
| *** placko__ has quit IRC | 15:43 | |
| samueldmq | where would htat script belong to ? ^ | 15:43 |
| *** placko_ has joined #openstack-infra | 15:43 | |
| *** moravec has joined #openstack-infra | 15:43 | |
| clarkb | devstack | 15:44 |
| *** bpokorny has joined #openstack-infra | 15:44 | |
| mordred | samueldmq: yah. you want to make sure devstack can be configured to use ldap, or federation or whatever | 15:44 |
| samueldmq | but devstack would install, let's say, openldap ? | 15:45 |
| samueldmq | seems like the wrong place to do it ? | 15:45 |
| mordred | it would totally install openldap | 15:45 |
| mordred | and configure it | 15:45 |
| mordred | same as it installs and configured mysql and rabbit | 15:45 |
| mordred | if you have a config variable set to say "keystone-use-ldap" | 15:45 |
| mordred | because you want it to be easy for developers to be able to set up a devstack that is configured to use openldap if that's the thing they're hacking on | 15:46 |
| samueldmq | okay, so adding the capability of installing dependencies for different keystone configurations is a thing for devstack | 15:46 |
| *** bcourt has quit IRC | 15:46 | |
| samueldmq | that is enabled thorugh a flag that is passed from project-config jobs | 15:46 |
| mordred | yah | 15:47 |
| samueldmq | clarkb: mordred yep, that makes sense; thanks for clarifying | 15:47 |
| mordred | \o/ | 15:47 |
| samueldmq | :) | 15:47 |
| samueldmq | mordred: also for things like #245304 (this is yours) | 15:47 |
| samueldmq | mordred: it'd be nice to have functional tests on it, wouldn't it ? | 15:47 |
| *** zns has quit IRC | 15:47 | |
| *** Sukhdev has joined #openstack-infra | 15:48 | |
| *** xyang1 has joined #openstack-infra | 15:48 | |
| mordred | totally | 15:48 |
| pabelanger | igorbelikov: will wait for jenkins, but so far looks good | 15:48 |
| samueldmq | mordred: ++ | 15:48 |
| mordred | samueldmq: adding tests is next on my list | 15:48 |
| mordred | samueldmq: I just didn't want to bother if people were opposed to the concept of the patc | 15:48 |
| clarkb | also there is already an attempt to make a two node federation install work woth a WIP patch to devstack gate, not sure where it got to | 15:48 |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr: Fail if no tests were successful executed https://review.openstack.org/246980 | 15:48 |
| samueldmq | mordred: makes sense to confirm before putting more effort, good for you | 15:48 |
| samueldmq | mordred: and for us since now people seem ot agree that's a good addition :) | 15:49 |
| *** dizquierdo has quit IRC | 15:51 | |
| *** smarcet1981 has joined #openstack-infra | 15:52 | |
| *** rkukura has joined #openstack-infra | 15:53 | |
| *** smarcet has quit IRC | 15:54 | |
| *** sabeen2 has quit IRC | 15:55 | |
| *** sabeen has joined #openstack-infra | 15:55 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Pin bashate to 0.3.1 https://review.openstack.org/236814 | 15:55 |
| *** mriedem is now known as mriedem_meeting | 15:56 | |
| *** kozhukalov_ has joined #openstack-infra | 15:56 | |
| katyafervent | Hi all | 15:56 |
| katyafervent | Could you please advise if this ok to use closes-bug and implements blueprint in one commit message? | 15:57 |
| *** moravec has quit IRC | 15:57 | |
| *** fhubik has quit IRC | 15:58 | |
| *** daemontool has quit IRC | 15:58 | |
| openstackgerrit | Mark T. Voelker proposed openstack-infra/irc-meetings: Adjust DefCore Meeting https://review.openstack.org/247000 | 15:58 |
| *** zns has joined #openstack-infra | 15:58 | |
| *** pradk_ has joined #openstack-infra | 15:58 | |
| *** daemontool has joined #openstack-infra | 15:59 | |
| *** yamahata has joined #openstack-infra | 15:59 | |
| *** dtantsur is now known as dtantsur|brb | 16:00 | |
| *** tdurakov_ has quit IRC | 16:01 | |
| *** tiswanso has quit IRC | 16:01 | |
| *** tdurakov_ has joined #openstack-infra | 16:01 | |
| *** dkranz has quit IRC | 16:01 | |
| *** tdurakov_ has quit IRC | 16:01 | |
| *** rbrndt has quit IRC | 16:01 | |
| *** tiswanso has joined #openstack-infra | 16:01 | |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Add listing of installed packages on undercloud node https://review.openstack.org/247002 | 16:03 |
| *** bcourt has joined #openstack-infra | 16:04 | |
| *** aspiers has joined #openstack-infra | 16:04 | |
| *** pcrews has joined #openstack-infra | 16:06 | |
| *** hemnafk is now known as hemna | 16:06 | |
| *** bswartz has quit IRC | 16:06 | |
| mordred | nibalizer: maybe we should land https://review.openstack.org/#/c/240459/ ? | 16:07 |
| openstackgerrit | Merged openstack-infra/ansible-puppet: Add tox -eansible-lint support https://review.openstack.org/244403 | 16:07 |
| anteaya | igorbelikov: thank you http://lists.openstack.org/pipermail/openstack-dev/2015-November/079769.html I appreciate the heads up | 16:08 |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: enable ebtables workaround in devstack https://review.openstack.org/247004 | 16:08 |
| *** erlon has quit IRC | 16:08 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: OIDC - OpenId Connect Implementation https://review.openstack.org/198458 | 16:09 |
| *** rcernin has quit IRC | 16:09 | |
| *** maishsk has quit IRC | 16:10 | |
| *** wuhg has quit IRC | 16:11 | |
| mordred | sdague: wow | 16:12 |
| *** maishsk has joined #openstack-infra | 16:12 | |
| *** mkwiek07 has quit IRC | 16:13 | |
| *** jaypipes has quit IRC | 16:13 | |
| *** elemoine has joined #openstack-infra | 16:13 | |
| sdague | mordred: yeh, it's a thing | 16:13 |
| sdague | it's not a pretty thing | 16:14 |
| sdague | we've sort of known about it for a while, but it's been really bad the last couple of months, and not sure why | 16:14 |
| mordred | sdague: this is another one of those things where I wish we had a systemic solution for openstack to be able to say "openstack depends on libvirt>=1.2.1" | 16:14 |
| *** Guest88694 has quit IRC | 16:14 | |
| sdague | mordred: sure | 16:14 |
| mordred | it's actually a _better_ example of that than things in the past which have been about getting access to new features for nfv work | 16:15 |
| *** ParsectiX has joined #openstack-infra | 16:16 | |
| *** dimtruck is now known as zz_dimtruck | 16:16 | |
| sdague | yeh, we're telling everyone they should update in our release notes - https://review.openstack.org/#/c/246580/ | 16:16 |
| jeblair | sdague: +2; do you want me to +3 or wait for test results? | 16:16 |
| mordred | ++ | 16:16 |
| sdague | we can wait for test results, we need the devstack patch approved as well | 16:16 |
| mordred | jeblair: don't miss the devstack patch too | 16:16 |
| mordred | jeblair: also, good morning! | 16:16 |
| *** maishsk_ has joined #openstack-infra | 16:17 | |
| jeblair | that will take slightly longer to review :) | 16:17 |
| sdague | I tested it for unconditional replacement, but dansmith was going to burn me at the stake for it | 16:17 |
| igorbelikov | anteaya: np, thanks for posting to openstack-dev:) just to be clear, the main reason to delay upgrade is a rollback plan? | 16:17 |
| *** dkranz has joined #openstack-infra | 16:17 | |
| sdague | so I made it an opt in param | 16:17 |
| sdague | and dansmith was probably right in objecting | 16:17 |
| clarkb | and its the sort of thing where if flock times out and fails thats no worse than the current thing | 16:17 |
| *** amitgandhinz has joined #openstack-infra | 16:18 | |
| clarkb | which is fail without trying | 16:18 |
| *** burgerk has joined #openstack-infra | 16:18 | |
| jeblair | mordred: re gertty request -- i agree, but 2 things make that tricky -- 1) depends-on is an openstackism (more or less), so needs to be configurable; 2) that introduces multiple parents which is more difficult to represent graphically, so will require some imagination and ascii-artistry. :) | 16:18 |
| anteaya | igorbelikov: well right now we have discovered that our git files and db dump we had for testing are asyncrounous so the testing isn't accurate | 16:18 |
| mordred | sdague: do you know about https://www.debian.org/doc/debian-policy/ap-pkg-diversions.html ? | 16:18 |
| anteaya | igorbelikov: so our rollback plan may indeed work however we haven't tested it in an environment we can trust | 16:18 |
| mordred | sdague: I know it's just a debian thing - but perhaps adding a dpkg-divert call if you're on a debian system would help make this safer for debian/ubuntu-based devstack users | 16:18 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Fix to prevent mantain unexistent users sessions: https://review.openstack.org/247010 | 16:19 |
| *** wolsen has joined #openstack-infra | 16:19 | |
| anteaya | igorbelikov: so a fault in our testing was discovered and we need time to re-evaluate | 16:19 |
| jeblair | mordred: we might be getting into lipstick on a pig territory :) | 16:19 |
| mordred | jeblair: nod on multiple-parents | 16:19 |
| *** edmondsw has quit IRC | 16:19 | |
| mordred | jeblair: indeed | 16:19 |
| sdague | mordred: I thought there was something like that | 16:19 |
| mordred | jeblair: that's why I didn't say itin the review - not sure how important the lipstick is | 16:19 |
| *** maishsk has quit IRC | 16:20 | |
| *** maishsk_ is now known as maishsk | 16:20 | |
| jeblair | mordred: what's really important to get right though is the color | 16:20 |
| *** mpavone has quit IRC | 16:20 | |
| sdague | yeh, I think we could make the work around better in the future, it would be good to get enough test results to ensure this actually fixes things | 16:20 |
| *** fawadkhaliq has quit IRC | 16:20 | |
| anteaya | igorbelikov: the db dump was created while gerrit was running so files were being created while other states were being saved, hence the asyncrous result | 16:20 |
| *** placko_ has quit IRC | 16:21 | |
| sdague | nova is going to ship instructions on how to do this for mitaka, maybe we'd add that there | 16:21 |
| sdague | for people that can't upgrade libvirt (for whatever reason) | 16:21 |
| sdague | it will be instructions though, not a script | 16:21 |
| jeblair | ++ | 16:21 |
| mordred | jeblair: ++ | 16:21 |
| clarkb | nova could install an entrypoint script thing that is optionally used | 16:21 |
| mordred | fwiw - rpm does not have a feature similar to dpkg-divert | 16:22 |
| clarkb | (I think setuptools can do that) | 16:22 |
| mordred | the closest rpm has is alternatives, which all packages participating in have to be aware of | 16:22 |
| mordred | so you can't divert out from under a package without the package maintainer knowin about it | 16:22 |
| *** dims_ has joined #openstack-infra | 16:22 | |
| sdague | so, honestly, this will be instructions from upstream. If a distro wants to bake that in, fine | 16:23 |
| sdague | but we're not going to have the upstream code actually do that | 16:23 |
| mordred | totally | 16:23 |
| *** flip214 is now known as flip211 | 16:23 | |
| mordred | I was more suggesting divert as a possible addition to the devstack patch - but since there is not an rpm equiv, it's not that worth it | 16:24 |
| mordred | since it still would not be safe to just always enable | 16:24 |
| jeblair | (the graphite whisper resize is still ongoing) | 16:24 |
| sdague | yeh, honestly, centos7 has a new enough version I think | 16:24 |
| *** dims has quit IRC | 16:24 | |
| sdague | as do all supported fedoras | 16:24 |
| mordred | jeblair: cool. btw - the devstack time graphs still look yuck | 16:24 |
| sdague | which is why I didn't add it in that path | 16:24 |
| igorbelikov | anteaya: got it, thanks. So this is an issue with the testing instance, not some bug you found in gerrit itself. We're thinking about upgrading our gerrit instance to 2.11 too and I'm just curious if any outstanding issues were uncovered during your testing) | 16:24 |
| *** rpodolyaka1 has quit IRC | 16:25 | |
| mordred | sdague: oh! well then in that case, ou could totally do the divert and enable it always in devstack, since that wold be safe, then the flag would not be needed | 16:25 |
| clarkb | igorbelikov: openid post auth redirection is currently broken | 16:25 |
| anteaya | igorbelikov: correct an issue with our testing data | 16:25 |
| jeblair | mordred: yes, they won't stop being yuck until at least the whisper resize hits them; i can check and verify if that's happened (though if it has, i doubt i'll be able to tell if it was 2 days ago or 2 minutes ago) | 16:25 |
| clarkb | but otherwise ya it was just testing data that needed to be redone | 16:25 |
| sdague | mordred: right, except people still object to use touching /sbin by default | 16:25 |
| mordred | jeblair: nah. that's cool- wasn't sure if new information was supposed to look good and bad yuck was just for history | 16:25 |
| *** yfried has quit IRC | 16:25 | |
| sdague | and I can understand that | 16:26 |
| mordred | sdague: seriously? in devstack? | 16:26 |
| igorbelikov | clarkb: is there a link with more details by any chance? | 16:26 |
| *** yamamoto has joined #openstack-infra | 16:26 | |
| clarkb | igorbelikov: on what? | 16:26 |
| sdague | mordred: yeh | 16:26 |
| mordred | wow | 16:26 |
| anteaya | igorbelikov: https://etherpad.openstack.org/p/test-gerrit-2.11 | 16:26 |
| mordred | those people have the entirely wrong idea about what devstack is | 16:26 |
| *** rpodolyaka1 has joined #openstack-infra | 16:26 | |
| sdague | mordred meet dansmith | 16:26 |
| anteaya | igorbelikov: and you know if you felt like helping us that would be great too | 16:26 |
| sdague | I'm staying out of it from here on out | 16:26 |
| *** ccrouch1 has joined #openstack-infra | 16:26 | |
| mordred | hey dansmith - I believe you have the wrong idea about what devstack is and should be. also, good morning! | 16:26 |
| *** ccrouch has quit IRC | 16:27 | |
| igorbelikov | anteaya: thanks. And I'd love to help | 16:27 |
| jeblair | i think in this particular case, opt-in is probably a good idea anyway. we expect it to be a temporary fix, and it will nice to be able to easily disable it later. | 16:27 |
| anteaya | igorbelikov: great go over that etherpad, any issue that is still an issue needs to be addressed, like the openid post auth redirection that is currently broken as clarkb said | 16:27 |
| jeblair | (regardless of whatever policy we may or may not think we have with sbin) | 16:28 |
| mordred | dansmith: I would suggest that given that devstack is and always has been accompanied with the warning "under no circumstances should you run this on anything real" - and since it by default mucks with kernel modules, installs and removes software - being protective of /sbin is misplaced | 16:28 |
| mordred | although I agree with jeblair that in this case making this thing optional is a fine idea | 16:28 |
| *** rbrndt has joined #openstack-infra | 16:28 | |
| dansmith | mordred: because I don't want it screwing with my /sbin? I get that devstack is supposed to obliterate your system to make things work | 16:28 |
| mordred | yah | 16:28 |
| *** edmondsw has joined #openstack-infra | 16:28 | |
| sdague | it would be a new threshold of monkeying with your system | 16:28 |
| mordred | why? | 16:29 |
| sdague | we've never modified sbin before | 16:29 |
| mordred | it instlals kernel modules | 16:29 |
| mordred | so | 16:29 |
| mordred | ? | 16:29 |
| mordred | sbin isn't that special | 16:29 |
| dansmith | etc is meant to be modified and configured | 16:29 |
| clarkb | jeblair: I haven't been paying much attention because gerrit and logstash but are we getting a feel for how bluebox and ovh are doing? | 16:29 |
| igorbelikov | anteaya: ok, I'll se what I can do | 16:29 |
| dansmith | packages expect you to monkey with etc | 16:29 |
| jeblair | clarkb: not really -- looking into that was what sent me into the graphite hole | 16:29 |
| clarkb | jeblair: gotcha | 16:29 |
| mordred | yeah. I understand that for everything in the world that isn't devstack | 16:29 |
| sdague | anyway, I think what's there is hopefully a reasonable compromise. I'm fine with the flag | 16:29 |
| mordred | but devstack is not a production tool | 16:29 |
| mordred | and should NEVER be used on a production system you care about - or even a test system you care about | 16:30 |
| dansmith | mordred: no, it's a dev tool and as a dev, any time it makes me blow my machine away to get it working again makes me mad | 16:30 |
| clarkb | sdague: dansmith semi related, but does neutron need to grow support for this too? | 16:30 |
| clarkb | and if so is that being done? | 16:30 |
| dansmith | clarkb: I dunno | 16:30 |
| clarkb | for --concurrent | 16:30 |
| sdague | clarkb: also I don't know | 16:30 |
| sdague | my guess is yes | 16:30 |
| clarkb | ok because i am 99% sure neutron touches ebtables too | 16:30 |
| clarkb | so if you fix nova and libvirt but not neutron we will still be broken | 16:31 |
| * persia wonders about shipping devstack as an OS image constructed with dib, rather than expecting anyone to "install" it, to avoid this class of debate | 16:31 | |
| *** Hal has joined #openstack-infra | 16:31 | |
| mordred | dansmith: I was unaware that we'd ever made any assertions that a system would be usable after a devstack run - and pretty much have always asserted that devstack should be run on special-purpose machines or vms | 16:31 |
| mordred | persia: booting os images is WAY harder than running stack.sh | 16:31 |
| *** Hal is now known as Guest90889 | 16:31 | |
| dansmith | mordred: yeah that's very true and it sucks | 16:31 |
| persia | mordred: `git clone foo; vagrant up` | 16:31 |
| mordred | dansmith: k. I mean, if that wasn't the case, I'd totally agree with you | 16:31 |
| sdague | clarkb: yes, it appears neutron is also impacted, I'll add them to the bug | 16:31 |
| dansmith | mordred: I don't do my dev in the cloud, nor do I want to, and nor do I want to have to even boot a new instance every time I need to run devstack | 16:31 |
| mordred | dansmith: but since it is, protecting part of the system when we trash the rest seems like too little too late | 16:32 |
| mordred | dansmith: you run devstack in a not-vm directly on your laptop? | 16:32 |
| sdague | also, if anyone knows netfilter people and can kindly ask them about the implications of cooperative opt in on non racing conditions, that would be swell | 16:32 |
| dansmith | mordred: no, that would be stupid | 16:32 |
| mordred | dansmith: ok. just making sure I hadn't missed a giant shift in the world :) | 16:32 |
| mordred | persia: vagrant does not work for me | 16:32 |
| dansmith | mordred: I run it on a dedicated physical machine with a ton of CPU and memory, which I can push a button and reimage on a whim | 16:33 |
| mordred | persia: and has never worked with any reliability | 16:33 |
| *** annegentle has quit IRC | 16:33 | |
| persia | Fair :) | 16:33 |
| dansmith | mordred: but it takes 30 minutes to do it and I almost only ever have to do it when devstack screws something up beyond what I can figure out how to undo | 16:33 |
| dansmith | mordred: which is okay, and why I have the big red button | 16:33 |
| mordred | dansmith: that sounds awesome. since you can reimage it - why would devstack breaking /sbin be any worse than devstack mucking with kernel modules and other kernel memory | 16:33 |
| mordred | ah | 16:33 |
| mordred | ok | 16:33 |
| dansmith | mordred: but I don't want to have to do that on every run | 16:33 |
| *** annegentle has joined #openstack-infra | 16:33 | |
| sdague | right, I think it's a fine safety valve for devstack on real hardware | 16:34 |
| jeblair | mordred: resize has not hit it yet | 16:34 |
| mordred | dansmith: my assumption is that nuking machines and restarting from scratch is a 5 minute task at most | 16:34 |
| sdague | mordred: really? the fastest I can get preseed down to is about 20 minutes for a full ubuntu install, and that's with a local mirror | 16:34 |
| dansmith | mordred: it takes me at least five minutes to boot my machine | 16:35 |
| dansmith | mordred: it's an hp rackmount box and it spends five minutes counting dimms and scsi drives | 16:35 |
| *** dprince has quit IRC | 16:35 | |
| dansmith | mordred: if it was an ibm box, that'd be about 20 minutes to bootloader :) | 16:35 |
| jeblair | dansmith: (i think there's an option in the hp bios to count the ram faster) | 16:35 |
| dansmith | I know, because, well you know :) | 16:35 |
| dansmith | jeblair: shush :) | 16:35 |
| mordred | becaus that's about how much time it takes for ironic to nuke a machine - 6 if you've got a particulalry slow POST | 16:35 |
| mordred | but I suppose not everyone has drunk the ironic cool-aid yet - so fair | 16:35 |
| mordred | oh! I guess you've got to re-run destack from scratch at that point | 16:35 |
| mordred | ok. I gotcha | 16:35 |
| dansmith | mordred: if this was majorly complicating something in devstack, I'd be on your side | 16:36 |
| mordred | WOW I HAVE 86 SECONDS OF LAG! | 16:36 |
| dansmith | mordred: but this is less work than the argument we're having, and it's purely focused at dev people, which is who devstack is supposed to help | 16:36 |
| *** EricGonc_ has quit IRC | 16:36 | |
| jeblair | "meetings are work product" | 16:37 |
| sdague | yeh, if the argument keeps going, we might even have test results before it's done | 16:37 |
| mordred | dansmith: sure. I just was concerned about a policy or line of thinking that devstack should avoid touching /sbin because somehow /sbin was precious | 16:37 |
| mordred | which I do not think is accurate compared to all of the other crazy things devstack already does. however, like I said, in this case, sure, it's fine to do the thing that sdague did | 16:37 |
| clarkb | mordred: because you wanted to know more about our storms. The had to close I84 between troutdale and cascade locks | 16:37 |
| mordred | clarkb: oh wow! | 16:38 |
| clarkb | mordred: apparently too much wind and trees on the freeway | 16:38 |
| dansmith | mordred: well, it is precious for the above stated reasons of not wanting you dicking with it without asking, but I think that (a) I agree on the goal of devstack and (2) we just disagree on practicalities | 16:38 |
| jeblair | clarkb: because of a particularly dark cloud? | 16:38 |
| *** Piet has joined #openstack-infra | 16:38 | |
| mordred | dansmith: I agree with your a and 2 points | 16:38 |
| dansmith | lol | 16:38 |
| dansmith | oops | 16:38 |
| jeblair | 1b is a non starter | 16:39 |
| mordred | dansmith: and given that "if this was majorly complicating something in devstack, I'd be on your side" I think I'm not worried about a hardline stance about a thing, so I'll say we pretty much agree | 16:39 |
| clarkb | "20 downed trees" | 16:39 |
| clarkb | I recommend not running into 20 trees while driving on the freeway | 16:39 |
| *** tiswanso has quit IRC | 16:39 | |
| mordred | clarkb: s/20/any/ | 16:39 |
| dansmith | clarkb: the wind took my interwebs away yesterday | 16:40 |
| jeblair | clarkb: what's your recommended number of trees to run into? | 16:40 |
| mordred | sdague: oh - and I was not talking about preseed - I think iso/preseed in this day and age is a complete and utter waste of time | 16:40 |
| clarkb | jeblair: best practice is 0 | 16:40 |
| greghaynes | ya, it got a bit stormy there | 16:40 |
| *** tiswanso has joined #openstack-infra | 16:40 | |
| jeblair | we had some lovely clouds at sunset | 16:41 |
| sdague | mordred: sure, though it gives me an environment I know other people will have | 16:41 |
| sdague | which, for testing devstack, is useful | 16:41 |
| mordred | sdague: those people are starting from a crazy-stance and should be educated, not coddled | 16:41 |
| mordred | but I hear you | 16:42 |
| greghaynes | we should get our dib images published | 16:42 |
| dansmith | sdague: you and I are crazy and monty is the sane one? didn't see that coming. | 16:42 |
| mordred | greghaynes: ++ | 16:42 |
| sdague | believe me, I've been through this whole image vs. auto-installer thing before | 16:42 |
| greghaynes | then you would really have an env others have | 16:42 |
| mordred | dansmith: some days are weird | 16:42 |
| sdague | https://github.com/finley/SystemImager/wiki | 16:42 |
| mordred | sdague: but these days, what with docker and vagrant and clouds | 16:42 |
| clarkb | our vhd images sort of already are published | 16:42 |
| clarkb | we would just have to flip the rax bit to cdn them then convince people to not lol at vhds | 16:43 |
| mordred | sdague: pretty much everyone under the sun has gotten used to the idea of simple base OS tat you can specialize with scripts and config-mgmt | 16:43 |
| sdague | I know, which is why 80% of dockerhub has heartbleed :) | 16:44 |
| * sdague ducks | 16:44 | |
| dansmith | lol | 16:44 |
| greghaynes | clarkb: no amound of convincing could make me not lol at vhd | 16:44 |
| mordred | greghaynes: you should never not lol at vhd | 16:44 |
| *** ashleighfarnham has joined #openstack-infra | 16:44 | |
| jeblair | (isn't that what a preseed file does?) | 16:44 |
| notmorgan | dansmith: whoa, did you just call monty sane? | 16:45 |
| mordred | notmorgan: it's a strange morning | 16:45 |
| *** tiswanso has quit IRC | 16:45 | |
| *** ashleighfarnham has quit IRC | 16:45 | |
| notmorgan | mordred: did I wake up in wonderland? | 16:45 |
| *** ashleighfarnham has joined #openstack-infra | 16:45 | |
| dansmith | notmorgan: no, I was sarcastically echoing his assertion of that fact, implying the opposite with my snark, of course | 16:45 |
| notmorgan | mordred: cause i suddenly feel like i did. | 16:45 |
| notmorgan | dansmith: ok too early sans coffee to parse that sentence | 16:45 |
| *** rpodolyaka1 has quit IRC | 16:45 | |
| dansmith | notmorgan: let me try again: "NO" | 16:46 |
| notmorgan | dansmith: LOL | 16:46 |
| mordred | :) | 16:46 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/246764 | 16:46 |
| * mordred will happily choose to avoid preseed and kickstart like the plague and with it also ignore anyone complaining that installing an OS takes any more than slightly longer than the machine POST cycle | 16:47 | |
| *** maishsk has quit IRC | 16:47 | |
| mordred | whichever way people want to assign sanity in that decision is up to them :) | 16:47 |
| notmorgan | oooh. | 16:47 |
| notmorgan | yeah. going to just opt to go get coffee instead of assigning sanity | 16:48 |
| *** tiswanso has joined #openstack-infra | 16:48 | |
| *** scheuran has quit IRC | 16:49 | |
| *** amuller is now known as amuller_afk | 16:49 | |
| *** sridhar_ram has joined #openstack-infra | 16:50 | |
| *** rpodolyaka1 has joined #openstack-infra | 16:52 | |
| *** otter768 has joined #openstack-infra | 16:53 | |
| *** jaypipes has joined #openstack-infra | 16:53 | |
| *** rpodolyaka1 has quit IRC | 16:53 | |
| *** fawadkhaliq has joined #openstack-infra | 16:54 | |
| *** fawadkhaliq has quit IRC | 16:55 | |
| *** fawadkhaliq has joined #openstack-infra | 16:55 | |
| *** rpodolyaka1 has joined #openstack-infra | 16:56 | |
| *** otter768 has quit IRC | 16:57 | |
| *** fedexo has joined #openstack-infra | 16:58 | |
| *** dtantsur|brb is now known as dtantsur | 16:59 | |
| *** stevemar_ has quit IRC | 16:59 | |
| *** dkranz has quit IRC | 16:59 | |
| *** tongli has joined #openstack-infra | 17:00 | |
| *** tdurakov_ has joined #openstack-infra | 17:00 | |
| *** tdurakov_ has quit IRC | 17:00 | |
| *** e0ne has quit IRC | 17:01 | |
| jeblair | clarkb, mordred: i've parsed the nodepool logs for runtimes; you can see them in nodepool.o.o:~corvus/runtimes2 if you want | 17:02 |
| jeblair | clarkb, mordred: short version is -- ovh is pretty consistently right between rax and hp, and bluebox is right around hp for almost all jobs | 17:03 |
| jeblair | clarkb, mordred: i've also spot checked failed jobs and don't see any failures that seem correlated with the environment | 17:04 |
| anteaya | nice | 17:04 |
| *** sshnaidm has quit IRC | 17:04 | |
| *** Sree has joined #openstack-infra | 17:04 | |
| jeblair | mostly ubuntu archive errors | 17:04 |
| asselin | jeblair, good morning. do you have time to discuss urllib3 insecure platform issues? | 17:05 |
| jeblair | ovh may have a slightly higher tendency to have the jenkins slave agent disconnect | 17:05 |
| *** diana_clarke has quit IRC | 17:05 | |
| *** abregman has joined #openstack-infra | 17:05 | |
| *** abregman is now known as abregman|home | 17:05 | |
| jeblair | and bluebox may be showing tendencies to uncover new race conditions in devstack-tempest jobs | 17:05 |
| jeblair | but i'm thinking that we can probably turn up the dial... maybe bump them both to 20? | 17:06 |
| jeblair | asselin: sure | 17:06 |
| * anteaya is good with 20 | 17:06 | |
| openstackgerrit | David TARDIVEL proposed openstack-infra/project-config: Add pypi publishing job for watcher. https://review.openstack.org/247043 | 17:06 |
| *** jlanoux has quit IRC | 17:06 | |
| mordred | jeblair: I'm good with that | 17:06 |
| *** kozhukalov_ has quit IRC | 17:07 | |
| *** unicell has joined #openstack-infra | 17:07 | |
| *** matrohon has quit IRC | 17:07 | |
| *** unicell1 has quit IRC | 17:07 | |
| *** rossella_s has quit IRC | 17:07 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Increase OVH and bluebox to 20 servers each https://review.openstack.org/247044 | 17:07 |
| jeblair | clarkb, mordred: ^ | 17:07 |
| jeblair | anteaya: ^ | 17:07 |
| openstackgerrit | Dirk Mueller proposed openstack-infra/project-config: Enable standard python jobs for pymod2pkg https://review.openstack.org/247045 | 17:07 |
| *** rossella_s has joined #openstack-infra | 17:07 | |
| asselin | jeblair, so I tried to resolve it correctly by using this: http://urllib3.readthedocs.org/en/latest/security.html#openssl-pyopenssl | 17:08 |
| *** mriedem_meeting is now known as mriedem | 17:08 | |
| jeblair | asselin: do you have a patch? | 17:08 |
| jeblair | asselin: oh no, you definitely don't want to do that | 17:09 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Change uuid columns to bigint https://review.openstack.org/235009 | 17:09 |
| jeblair | asselin: we just want to shut up the warnings | 17:09 |
| asselin | jeblair, I didn't submit upstream as I wasn't sure it's something needed. | 17:09 |
| asselin | jeblair, ok I see, then I will revert and shut up the warnings | 17:10 |
| mtreinish | mordred: ^^^ if you can muster up the courage to review that it would be awesome | 17:10 |
| jeblair | asselin: you might take a look at the requestsexceptions library :) | 17:10 |
| *** sileht has quit IRC | 17:10 | |
| *** sileht has joined #openstack-infra | 17:11 | |
| asselin | jeblair, but it sounds like its something worth to have upstream as well | 17:11 |
| nibalizer | mordred: approved :) | 17:11 |
| *** jistr has quit IRC | 17:11 | |
| mtreinish | nibalizer: do you think you could help with: https://etherpad.openstack.org/p/the-big-subunit2sql-migration when the time is right? | 17:12 |
| asselin | jeblair, yeah I did have a working version of just the warnings disabled. I will switch back to that | 17:12 |
| *** diana_clarke has joined #openstack-infra | 17:12 | |
| nibalizer | mtreinish: i'd be happy to | 17:12 |
| nibalizer | when is that time | 17:12 |
| *** dzamboni has quit IRC | 17:13 | |
| *** asadoughi has joined #openstack-infra | 17:13 | |
| *** asadoughi has left #openstack-infra | 17:14 | |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Bump oslo.versionedobjects>=0.13.0 https://review.openstack.org/245990 | 17:15 |
| clarkb | jeblair bumping dial sounds great will revuew in a few | 17:16 |
| mtreinish | nibalizer: well any time after those 3 patches merge | 17:17 |
| *** ociuhandu has quit IRC | 17:17 | |
| mtreinish | the only manual things are for me to push the release, and stopping puppet and the worker | 17:17 |
| mtreinish | oh and I guess starting things back up again after the migration is done (which will probably be several days to run) | 17:17 |
| *** moravec has joined #openstack-infra | 17:18 | |
| *** hashar has quit IRC | 17:18 | |
| *** kozhukalov_ has joined #openstack-infra | 17:19 | |
| *** daemontool has quit IRC | 17:20 | |
| *** fitoduarte has joined #openstack-infra | 17:20 | |
| *** gyee has joined #openstack-infra | 17:20 | |
| zaro | morning | 17:21 |
| mordred | mgagne: our new internap account has no networks in it | 17:21 |
| mordred | mgagne: should I just be patient? | 17:21 |
| mgagne | mordred: go in portal and activate a region | 17:22 |
| nibalizer | mordred: when are you looking to do it then? | 17:22 |
| nibalizer | mordred: mgagne awesome to get more quota! | 17:22 |
| *** Hal has joined #openstack-infra | 17:22 | |
| *** Hal is now known as Guest25643 | 17:22 | |
| openstackgerrit | Merged openstack-infra/system-config: Add puppetdb_file report processor https://review.openstack.org/240459 | 17:22 |
| mgagne | mordred should be here: https://login.internap.com/ | 17:23 |
| *** dguitarbite has quit IRC | 17:23 | |
| mgagne | mordred I have access to so many login pages, hard to find the right one :-/ | 17:23 |
| mgagne | mordred should be on that page: https://login.internap.com/devices/agile bottom right | 17:23 |
| *** dguitarbite has joined #openstack-infra | 17:24 | |
| mordred | mgagne: oh - I mena, I can log in to horizon for it just great | 17:24 |
| anteaya | morning zaro | 17:24 |
| anteaya | zaro: what do you want to work on first? I | 17:24 |
| *** abregman|home is now known as abregman|afk | 17:24 | |
| anteaya | 'd like to help you | 17:24 |
| mordred | mgagne: OH! | 17:24 |
| mgagne | mordred you need to "activate" the region so we provision networks for you | 17:24 |
| mordred | mgagne: I need to activate | 17:24 |
| mordred | sorry, I forgot | 17:24 |
| *** Guest90889 has quit IRC | 17:25 | |
| *** angdraug has quit IRC | 17:25 | |
| mgagne | mordred we don't hand out IPv4 to everyone registering for our services for obvious reasons | 17:25 |
| zaro | anteaya: only 2 things with regards to upgrade: rollback testing and URL with '//' on review-dev. | 17:26 |
| clarkb | mgagne: what about ipv6? | 17:27 |
| mgagne | mordred I'm still waiting for a reply to my email. Although we like to joke about our boss doing nothing, he is very busy doing "nothing" not at his desk and replying to emails =) | 17:27 |
| clarkb | (we can in theory operate on ipv6 only though I don't think we have tested this theory yet) | 17:27 |
| *** maishsk has joined #openstack-infra | 17:27 | |
| zaro | anteaya: i'm probably going to do both in parallel since 1st one takes long times and requires waiting around for reindex and what not | 17:27 |
| *** Sree has quit IRC | 17:27 | |
| asselin | jeblair, why just ignore the warning and not try to use the secure packages? | 17:28 |
| jeblair | asselin: because it's insane | 17:28 |
| mgagne | clarkb it would involve a lot more people than our team and we don't have short-term plan to support it. I'll see what can be done. | 17:29 |
| *** changbl has joined #openstack-infra | 17:29 | |
| clarkb | mgagne: its not required just curious if it is an option | 17:29 |
| zaro | anteaya: thanks for the offer but i think since upgrade has been postpone we have lots of time now. i have time to manage | 17:29 |
| *** Ala has quit IRC | 17:29 | |
| clarkb | mgagne: since I know sometimes when people have trouble with ipv4 they say "what about ipv6 instead" | 17:29 |
| mgagne | clarkb sure. it's unfortunately not an option as of today | 17:29 |
| jeblair | yeah, we just need instances that are on the internet, we don't care which internet :) | 17:30 |
| *** ildikov has joined #openstack-infra | 17:30 | |
| *** amrith is now known as _amrith_ | 17:30 | |
| mgagne | jeblair I'm sure the amount of IPv4 required will force our team and network admin to consider IPv6 | 17:30 |
| jeblair | mgagne: we're helping already! | 17:31 |
| mordred | mgagne: :) | 17:31 |
| *** Sukhdev has quit IRC | 17:31 | |
| nibalizer | ah I meant mtreinish when are you looking to do it? | 17:31 |
| *** stevemar_ has joined #openstack-infra | 17:31 | |
| jeblair | mgagne, mordred: which region are we using? | 17:31 |
| anteaya | zaro: okay keep me posted on what I can do to help test things, like make new patches when the new db env is up | 17:32 |
| pleia2 | good morning | 17:32 |
| mordred | jeblair: I'd say maybe we should start using dallas? | 17:32 |
| mordred | jeblair: since infra control plane is there | 17:32 |
| openstackgerrit | Merged openstack-infra/shade: Add test for os_user Ansible module https://review.openstack.org/245875 | 17:32 |
| *** jordanP has quit IRC | 17:33 | |
| anteaya | morning pleia2 | 17:33 |
| mordred | jeblair: and we can add ams and nj later? OR - maybe mgagne has a better idea | 17:33 |
| clarkb | or use !dallas for better geographical diversity | 17:33 |
| jeblair | mgagne: did you have a preference? istr something about montreal in our conversation...? | 17:33 |
| clarkb | (granted control plane) | 17:33 |
| *** unicell has quit IRC | 17:33 | |
| *** gordc has joined #openstack-infra | 17:33 | |
| *** tonytan4ever has quit IRC | 17:33 | |
| *** tsg has joined #openstack-infra | 17:34 | |
| jroll | can someone teach me to graphite? trying to learn about the rate of ironic patchsets (e.g. x/hour | 17:34 |
| jroll | ) | 17:34 |
| mtreinish | nibalizer: hopefully sometime next week all the conditions will be met | 17:34 |
| jeblair | i love that the standard .openrc we get from every openstack cloud has 12 lines of comments explaining how "adding keystone" changes things, and that we've standardized on the term "tenant". | 17:35 |
| mgagne | jeblair this has yet to be defined. Main point is we wish to be able to easily provision new and future hardware | 17:35 |
| mordred | jeblair: that is actually fixed in latest horizon, fwiw | 17:35 |
| clarkb | jroll: we have patchsets per $time in graphite but not on a per project basis I don't think | 17:35 |
| mordred | jeblair: I went to make a rage-patch about that a little while ago | 17:35 |
| jeblair | mordred: thanks :) | 17:35 |
| clarkb | jroll: if you want per project data like that you may have to ask gerrit for a listing over the last month and do maths | 17:35 |
| jroll | clarkb: runs of a particular job would be close enough for me, which we appear to maybe have? | 17:35 |
| clarkb | jroll: yes we keep track of runs of jobs | 17:36 |
| jeblair | jroll: which job? | 17:36 |
| jroll | clarkb: so, right now I'm looking at stats.zuul.pipeline.check.openstack.ironic.total_changes | 17:36 |
| jroll | but aggregation things aren't being helpful | 17:36 |
| *** Sukhdev has joined #openstack-infra | 17:36 | |
| mgagne | jeblair yes, montreal was in the conversation. We have an iWeb region in Montreal which isn't available through the API for all sort of reasons I'll spare you. We also have another Internap region in Montreal for a beta we made last year for baremetal. | 17:36 |
| mgagne | jeblair We have yet to decide if we will keep it or not as there would be a lot of stuff to do to make it a non-beta region. | 17:36 |
| jroll | jeblair: check-tempest-dsvm-ironic-pxe_ipa would work | 17:37 |
| jeblair | jroll: oh, that total_changes might work | 17:37 |
| clarkb | (also I thought russellb's review stats crunched out these numbers but I guess thats no longer running?) | 17:37 |
| lifeless | mordred: bug me about ? | 17:37 |
| mordred | lifeless: GUH ... there was a ... | 17:37 |
| anteaya | does anyone have time to take a peek at javamelody and see what gerrit is doing vis a vis garbage collection? http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=25&rra_id=all | 17:37 |
| mordred | OH | 17:37 |
| jroll | jeblair: so, the graph I get from that has data points of 0.xxx, is that per-minute/hour/? | 17:37 |
| mgagne | mordred still waiting for UX team to upgrade our Horizon theme. Ours isn't compatible with kilo yet. | 17:38 |
| *** mwagner has quit IRC | 17:38 | |
| nibalizer | mtreinish: ok cool | 17:38 |
| mgagne | mordred I'm sure you can install your own inception version within our cloud and customize it to your taste =) | 17:38 |
| clarkb | anteaya: it is not GCing like it does when it gets sad | 17:38 |
| anteaya | clarkb: I see russellb's site updated 15:56 utc today | 17:39 |
| *** elemoine has quit IRC | 17:39 | |
| anteaya | clarkb: thanks for looking | 17:39 |
| clarkb | anteaya: oh where is russellb's stuff hosted? | 17:39 |
| clarkb | jroll: ^ russellb may just have an answer for you | 17:39 |
| anteaya | http://russellbryant.net/openstack-stats/infra-reviewers-365.txt | 17:39 |
| clarkb | greghaynes: https://review.openstack.org/#/c/197722/ failed testing | 17:39 |
| mtreinish | nibalizer: I wouldn't complain if it happened sooner though :) | 17:40 |
| greghaynes | clarkb: ugh, I must have derped merge resolution | 17:40 |
| jeblair | jroll: try "summarize(stats_counts.zuul.pipeline.check.openstack.ironic.total_changes, '1h')" | 17:40 |
| mtreinish | this should hopefully fix a lot of the performance issues we've been hitting on the db | 17:40 |
| greghaynes | clarkb: oh, but the parents all pass | 17:40 |
| *** mestery has joined #openstack-infra | 17:40 | |
| openstackgerrit | Merged openstack-infra/zuul: Simplify wait_time stats https://review.openstack.org/246630 | 17:40 |
| greghaynes | clarkb: that is scary | 17:40 |
| clarkb | greghaynes: the one parent does ya | 17:41 |
| anteaya | jroll: you can also run it yourself in case russellb isn't tracking what you want: http://git.openstack.org/cgit/openstack-infra/reviewstats/tree/README.rst | 17:41 |
| jeblair | jroll: i believe 'stats_counts' is the prefix you want for aggregation -- i think the value in 'stats' has been converted to a rate already | 17:41 |
| nibalizer | mtreinish: next week is good since I am traveling this week | 17:41 |
| clarkb | greghaynes: I am about to be looking at the nasty rebase diff | 17:41 |
| clarkb | greghaynes: for the parent | 17:41 |
| jroll | jeblair: nice, ty \o/ | 17:41 |
| jroll | yeah, makes sense | 17:41 |
| *** armax has joined #openstack-infra | 17:41 | |
| jroll | I haven't used statsd stuff in a bit :) | 17:41 |
| jeblair | jroll: so stats_counts provides a raw count which you can then feed to summarize to aggregate how you want | 17:41 |
| jroll | jeblair: right, cool | 17:42 |
| greghaynes | clarkb: looks like the devstack fail there was just a slow node | 17:42 |
| greghaynes | clarkb: so that is probably a race :( | 17:42 |
| jeblair | jroll: note this will give you rechecks too -- so strictly speaking, the gerrit query / reviewstats method is better data | 17:42 |
| jeblair | jroll: but if you want a mostly-accurate pretty graph quickly ... :) | 17:42 |
| jroll | jeblair: well, this is for third party CI "what's the volume we need to support" question | 17:43 |
| crinkle | greghaynes: could you look at https://review.openstack.org/#/c/231757/ to see if it makes sense? i can explain the puppety parts if needed | 17:43 |
| jeblair | mgagne: so which resion should we use? :) | 17:43 |
| jroll | jeblair: so that is to say, rechecks are helpful here :) | 17:43 |
| mgagne | jeblair in short: TBD | 17:43 |
| *** Swami has quit IRC | 17:44 | |
| * clarkb wonders if people have been restarting nodepool when making non test changes to nodepool | 17:44 | |
| greghaynes | crinkle: oh, youre actually commiting the static leasing - I was thinking we would use bifrost to gen that from inventory | 17:44 |
| greghaynes | crinkle: also, when everyone involved is a co-author, who can review? ;) | 17:44 |
| clarkb | there is a fairly large set of changes according to the diff I am reading that would be good to get in prod (to avoid a massive pileup of new things that could go wrong) | 17:44 |
| mgagne | jeblair I suggest nyj01 for now. | 17:44 |
| mgagne | jeblair as we don't have much capacity in mtl01 due to its beta status | 17:45 |
| crinkle | greghaynes: so i'm not sure how it would work with bifrost, because we also need bifrost to dhcp the 10 net addresses on eth2 | 17:45 |
| *** terryw is now known as otherwiseguy | 17:45 | |
| crinkle | greghaynes: bifrost isn't set up to dhcp on two interfaces | 17:45 |
| greghaynes | crinkle: Yep, bifrost would have to make two dhcp configs | 17:45 |
| crinkle | greghaynes: this seemed easier | 17:45 |
| jeblair | mgagne: ok. we'll be creating a region-local pypi mirror, so if we want to add/move regions, it will take us about 2 days to do so. in short, we can be flexble and are happy to move, but making our best guess for what will work for a while would be good too. :) | 17:45 |
| greghaynes | crinkle: I think even if it doesnt manage both dhcp configs, it could gen the static leasing dir and then we just commit the actual dhcpd config | 17:46 |
| mgagne | jeblair I'll poke my boss ASAP | 17:46 |
| mgagne | jeblair we shouldn't be guessing those infos | 17:46 |
| *** claudiub has quit IRC | 17:46 | |
| greghaynes | crinkle: basically I am worried that changing inventory is going to mean we need to remember where else to change rather than just update the inventory | 17:46 |
| clarkb | (nodepool was started on the 15th which is recent so I think we may have those changes in production) | 17:46 |
| *** tdurakov_ has joined #openstack-infra | 17:47 | |
| crinkle | greghaynes: i tried for kind of a long time to get bifrost to do that, this was a much shorter path to success. i'd rather spend time getting glean to work properly than getting bifrost to do weird things | 17:47 |
| greghaynes | crinkle: hrm, I thought I had bifrost doing that | 17:47 |
| crinkle | it does either static leases or dhcp-range, not both, and it doesn't accept more than one network interface | 17:48 |
| crinkle | greghaynes: maybe you can show me what you were doing | 17:48 |
| openstackgerrit | Merged openstack-infra/project-config: Increase OVH and bluebox to 20 servers each https://review.openstack.org/247044 | 17:48 |
| *** _amrith_ is now known as amrith | 17:48 | |
| greghaynes | crinkle: as a middle ground, even if we just had an ansible play that read the inventory and gen'd that dir. Basically I just want to avoid duplicating source of truth for that info somehow | 17:49 |
| greghaynes | crinkle: That actaully might be a better option, bypass bifrost entirely and write a little ansible play | 17:49 |
| crinkle | greghaynes: okay, that sounds reasonable | 17:50 |
| crinkle | greghaynes: if you got it to work on its own that would be better though | 17:51 |
| clarkb | yolanda: if you are around I am not sure https://review.openstack.org/#/c/192629/ is the best course of action for nodepool | 17:51 |
| greghaynes | crinkle: Yea, I am hoping you got that data off disk and didn't type it all in by hand ;) | 17:51 |
| clarkb | yolanda: the laternative is you add the provider back to nodepool, set max nodes to -1, let nodepool cleanup after itself then delete the provider | 17:52 |
| clarkb | yolanda: with that change in nodepool will wipe the DB forcing you to do all the cleanup manually | 17:52 |
| crinkle | greghaynes: ya i wrote some python to parse the inventory json :P | 17:52 |
| greghaynes | crinkle: oh! | 17:52 |
| greghaynes | crinkle: well, we can just use that | 17:52 |
| greghaynes | crinkle: anyhow, I can poke at that | 17:53 |
| greghaynes | crinkle: I need to finish writing up a spec real quick, so after lunch? | 17:53 |
| crinkle | greghaynes: kk | 17:53 |
| clarkb | yolanda: basically I think both the old case and new case are not great | 17:53 |
| clarkb | but with old case I never had to talk directly to a cloud | 17:54 |
| clarkb | greghaynes: I don't see any rebase issues at least in the first change | 17:55 |
| clarkb | so reapplying my +2 | 17:55 |
| greghaynes | clarkb: sweet | 17:56 |
| *** bswartz has joined #openstack-infra | 17:56 | |
| *** apoorvad has joined #openstack-infra | 17:56 | |
| greghaynes | clarkb: That race is totally unrelated AFAICT but it seems to be newish | 17:56 |
| greghaynes | so we must have merged a race recently | 17:56 |
| clarkb | greghaynes: ok its a race in the integration test though? | 17:56 |
| greghaynes | the dsvm fail was just a slow node | 17:57 |
| greghaynes | in the coverage run | 17:57 |
| clarkb | ah ok so race in coverage | 17:57 |
| *** tdurakov_ has quit IRC | 17:57 | |
| *** sarob_ has joined #openstack-infra | 17:57 | |
| *** tdurakov_ has joined #openstack-infra | 17:59 | |
| *** aspiers has quit IRC | 17:59 | |
| greghaynes | clarkb: we should probably document that provider cleanup deal | 18:00 |
| *** achanda has quit IRC | 18:00 | |
| clarkb | greghaynes: ya | 18:00 |
| clarkb | greghaynes: so all that test does is "boot" a vm setting the az flag | 18:02 |
| *** dramalho has joined #openstack-infra | 18:02 | |
| *** achanda has joined #openstack-infra | 18:02 | |
| jeblair | clarkb, yolanda: er, yes -- that's an _intentional_ leak of nodes from nodepool :( | 18:02 |
| clarkb | greghaynes: the node goes ready at http://logs.openstack.org/22/197722/32/check/nodepool-coverage/791e5d1/console.html#_2015-11-17_18_15_50_300 | 18:03 |
| *** dprince has joined #openstack-infra | 18:03 | |
| *** Swami has joined #openstack-infra | 18:03 | |
| jeblair | clarkb, yolanda: nodepool was designed specifically to avoid that case | 18:03 |
| *** watanabe_isao has joined #openstack-infra | 18:03 | |
| *** achanda has quit IRC | 18:03 | |
| yolanda | jeblair, clarkb, so this change had been on long time under review and i didn't get that feedback :( anyway, what i'd like to avoid is the situation that we had, where you end having to do a manual cleanup on database | 18:04 |
| yolanda | even having to manually set the -1, etc... doesn't look to me as the right way to disconnect a provider | 18:04 |
| clarkb | yolanda: what is wrong with it? it allows existing VMs to be used and cycle out, no new images will be built then you remvoe the provider | 18:05 |
| jeblair | yolanda: it's currently the only way that cleans everything up. | 18:05 |
| *** sabeen has quit IRC | 18:05 | |
| clarkb | (I agree with greghaynes that it could be better documented though) | 18:05 |
| *** moravec has quit IRC | 18:05 | |
| *** daemontool has joined #openstack-infra | 18:05 | |
| *** mestery has quit IRC | 18:05 | |
| greghaynes | Any solution is going to require you to specify provider info and then something to the effect of "put it in cleanup mode", the -1 solution seems kind of like the minimum info you could provide for that | 18:06 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Disable InsecurePlatform Warnings https://review.openstack.org/247069 | 18:06 |
| yolanda | even documented, or have some action in nodepool, some command to perform the disconnection | 18:06 |
| *** dramalho has quit IRC | 18:06 | |
| asselin | jeblair, thanks for the suggestion. keeping it simple ^^ | 18:06 |
| yolanda | greghaynes, i agree with that "put on cleanup mode" | 18:07 |
| openstackgerrit | JJ Asghar proposed openstack-infra/irc-meetings: Removed No longer attended OpsMeeting https://review.openstack.org/247070 | 18:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Revert "Allow to delete nodes with missing target or provider" https://review.openstack.org/247071 | 18:07 |
| *** moravec has joined #openstack-infra | 18:08 | |
| *** sarob_ has quit IRC | 18:08 | |
| *** sdake has quit IRC | 18:08 | |
| jeblair | =-1 is roughtly "put in cleanup mode" | 18:08 |
| jeblair | i guess we could have an option for that that uses words instead of numbers :) | 18:09 |
| yolanda | well, it requires manual intervention on a system that should be automated. And yes, that option is not clear or trivial | 18:09 |
| clarkb | greghaynes: so test_node_az is waiting for image then waiting for node, its not immediately clear which one we ended up blocking on but according ot log both should've transitioned to the state we want? | 18:09 |
| greghaynes | clarkb: Re: race - It must have not written the node state to the database? I kind of want to add a big of debug to our db polling in the tests that outputs node state each iteration | 18:09 |
| clarkb | greghaynes: that is one way it could happen | 18:10 |
| greghaynes | clarkb: It is blocking on db polling because the traceback from the timeout hits pymysql | 18:10 |
| *** dkranz has joined #openstack-infra | 18:10 | |
| clarkb | ah | 18:10 |
| jeblair | yolanda: er, that's not manual -- it's a change to the config file | 18:10 |
| jeblair | yolanda: no manual intervention is required, you're just changing nodepool's operating parameters in the way they are normally changed | 18:10 |
| greghaynes | clarkb: I think that makes sense, at least... | 18:10 |
| greghaynes | clarkb: it actually doesnt go up through the waitForNodes or anything though, so IMBW. more digging | 18:11 |
| yolanda | jeblair, ok, then at least some clear documentation is needed. If you look at history of the change, we hit that, and some user showed his appreciation of that change, because he ended on same situation | 18:11 |
| clarkb | greghaynes: oh its fialing in cleanup | 18:11 |
| *** tdurakov_ has quit IRC | 18:11 | |
| clarkb | greghaynes: it can't drop the database because something is holding it open? | 18:11 |
| greghaynes | clarkb: hah | 18:11 |
| clarkb | greghaynes: that says GC fail to me | 18:11 |
| *** tdurakov_ has joined #openstack-infra | 18:11 | |
| openstackgerrit | JJ Asghar proposed openstack-infra/irc-meetings: Added OSOps to the Monitoring and Tools WG https://review.openstack.org/247075 | 18:12 |
| clarkb | greghaynes: but ya it looks like its hanging trying to delete the db but it might also be an order of ops in cleanup issue | 18:12 |
| greghaynes | clarkb: it would be neat if the test case would debug "starting cleanup" | 18:12 |
| jeblair | yolanda: agreed | 18:12 |
| clarkb | greghaynes: that should be a simple addition if you want to add it | 18:12 |
| clarkb | greghaynes: but ya | 18:12 |
| clarkb | the -1 behavior should be documented fwiw | 18:13 |
| clarkb | its just that we don't explicitly call it out as "this is how you remove a provider cleanly" | 18:13 |
| greghaynes | clarkb: I wonder if our order of operations for fixture cleanup is out of wack | 18:13 |
| *** watanabe_isao has quit IRC | 18:13 | |
| *** sdake has joined #openstack-infra | 18:13 | |
| clarkb | greghaynes: thats a possibility | 18:13 |
| clarkb | greghaynes: cleanup calls should be a stack, so first in last out iirc | 18:14 |
| mordred | yup | 18:14 |
| clarkb | lifeless: ^ is that correct? | 18:14 |
| mordred | that is correct | 18:14 |
| clarkb | so DB should go first | 18:14 |
| clarkb | then it gets called last | 18:14 |
| greghaynes | hrm | 18:15 |
| *** mestery has joined #openstack-infra | 18:15 | |
| *** e0ne has joined #openstack-infra | 18:15 | |
| *** tonytan4ever has joined #openstack-infra | 18:15 | |
| lifeless | clarkb: cleansup are LIFO yes | 18:16 |
| *** fedexo has quit IRC | 18:16 | |
| *** yfried has joined #openstack-infra | 18:16 | |
| clarkb | looking at our stack I don't see anything htat should be a problem | 18:16 |
| *** daemontool has quit IRC | 18:16 | |
| clarkb | its basically make config file, fake out subprocess, setup db, setup gearman | 18:16 |
| *** signed8b_ has joined #openstack-infra | 18:17 | |
| greghaynes | Yea, there is definitely a lack of info here so kind of grasping at straws | 18:17 |
| clarkb | mordred: what sorts of thing would make drop database block in mysql? | 18:18 |
| lifeless | you could add attachments to your fixtures to describe what happens | 18:18 |
| lifeless | maybe you have threads? | 18:18 |
| *** unicell has joined #openstack-infra | 18:18 | |
| openstackgerrit | Dan Moravec proposed openstack-infra/subunit2sql: Add dates to x-axis for subunit2sql graphs https://review.openstack.org/246641 | 18:18 |
| greghaynes | we do have threads | 18:18 |
| lifeless | that are alive | 18:18 |
| *** Sukhdev has quit IRC | 18:19 | |
| greghaynes | It is a bit crazy they would block long enough for that timeout | 18:19 |
| clarkb | right | 18:19 |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 18:19 |
| mordred | clarkb: table lock or something similar | 18:19 |
| *** signed8bit has quit IRC | 18:19 | |
| clarkb | though the periodic cleanup and friends may be at fault | 18:20 |
| *** shashank_hegde has joined #openstack-infra | 18:20 | |
| greghaynes | clarkb: we call nodepool.stop before that fixture cleanup | 18:20 |
| *** dprince has quit IRC | 18:20 | |
| jeblair | mordred: thinking an A1.4 with a cinder volume for the mirror? | 18:20 |
| *** dprince has joined #openstack-infra | 18:20 | |
| clarkb | greghaynes: I am not sure if that is enough to kill the cron runs | 18:20 |
| *** regXboi has joined #openstack-infra | 18:20 | |
| jeblair | | ID | Name | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public | | 18:20 |
| jeblair | | A1.4 | A1.4 | 4096 | 80 | 0 | | 4 | 1.0 | N/A | | 18:20 |
| greghaynes | clarkb: haha | 18:20 |
| clarkb | greghaynes: nodepool registers them but does stop deregister? | 18:20 |
| mordred | jeblair: yah. I think I agree with that | 18:20 |
| greghaynes | clarkb: that is a *really* good question | 18:20 |
| *** regXboi has quit IRC | 18:21 | |
| mordred | jeblair: I should work on the mirror.region.cloud.openstack.org change at some point too | 18:21 |
| *** fawadkhaliq has quit IRC | 18:21 | |
| *** sarob_ has joined #openstack-infra | 18:22 | |
| *** dtantsur is now known as dtantsur|afk | 18:22 | |
| jeblair | mordred: yeah, i think that needs a small nodepool change | 18:22 |
| mordred | jeblair: yah | 18:23 |
| clarkb | what needs a nodepool change? | 18:23 |
| greghaynes | clarkb: http://logs.openstack.org/52/233852/3/gate/gate-nodepool-python27/3a65946/console.html.gz | 18:24 |
| greghaynes | clarkb: that is the race I hit last week | 18:24 |
| clarkb | the ready scripts need to be updated | 18:24 |
| *** ociuhandu has joined #openstack-infra | 18:24 | |
| greghaynes | clarkb: looks like its the sam eone | 18:24 |
| *** ihrachys has quit IRC | 18:24 | |
| clarkb | greghaynes: ya look ssimilar | 18:24 |
| jeblair | clarkb: nodepool needs to pass "cloud name" as a parameter to the ready scripts | 18:25 |
| mordred | clarkb: ^^ that | 18:25 |
| clarkb | ah | 18:25 |
| jeblair | s/parameter/env var/ | 18:25 |
| *** amuller_afk is now known as amuller | 18:25 | |
| jeblair | but yeah, that, then duplicate dns entries, then update the scripts, then rename the hosts, then drop the old entries. actually not too hard -- i've renamed one mirror already | 18:26 |
| clarkb | overall process is something like move pypi mirror to /pypi, redirect / to /pypi, update ready scripts to use /pypi path, make new A records, update ready script to use new names (this part requires the cloud to be passed in), remove old name, add extra mirrors | 18:26 |
| clarkb | I suppose you can do the host part before or after the pypi move | 18:26 |
| jeblair | clarkb: you are combining that with the pypi move | 18:27 |
| *** mwagner has joined #openstack-infra | 18:27 | |
| *** ayoung has quit IRC | 18:27 | |
| jeblair | it probably makes sense :) | 18:27 |
| clarkb | right they don't need to happen together or in a particular order | 18:27 |
| *** sarob_ has quit IRC | 18:27 | |
| *** zns has quit IRC | 18:27 | |
| jeblair | anyway, first step is get the nodepool change in prod | 18:28 |
| *** rpodolyaka1 has quit IRC | 18:28 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add internap pypi mirror https://review.openstack.org/247082 | 18:28 |
| *** sdake has quit IRC | 18:28 | |
| *** dprince has quit IRC | 18:29 | |
| *** dprince has joined #openstack-infra | 18:29 | |
| jeblair | stupid question, but can we do *.openstack.org in the list of acceptable mirrors for pip? | 18:29 |
| greghaynes | internap == even more clouds? | 18:30 |
| *** annegentle has quit IRC | 18:30 | |
| jeblair | greghaynes: ayup | 18:30 |
| greghaynes | awesome! | 18:30 |
| jeblair | greghaynes: mgagne is our contact there :) | 18:30 |
| *** ifarkas has quit IRC | 18:30 | |
| anteaya | I read an article about a study of street names, in the states the most popular is main followed by second street, first street became main street in many places | 18:31 |
| *** trown is now known as trown|lunch | 18:31 | |
| anteaya | in canada popular names are king, queen, princess | 18:31 |
| *** rpodolyaka1 has joined #openstack-infra | 18:31 | |
| anteaya | I'm guessing states went with numbers as a pushback against royal names | 18:31 |
| *** rpodolyaka1 has quit IRC | 18:31 | |
| *** shashank_hegde has quit IRC | 18:31 | |
| anteaya | but having everything named one or two gets confusing | 18:32 |
| *** rpodolyaka1 has joined #openstack-infra | 18:32 | |
| mtreinish | mordred, jeblair: might be of interest: http://i.imgur.com/I6yuHYe.png | 18:32 |
| mtreinish | fwiw the 100% is on 2 runs | 18:32 |
| * anteaya hopes for a graph with many labels | 18:33 | |
| mtreinish | anteaya: sorry | 18:33 |
| clarkb | anteaya: I really like portlands system north south running streets are numbered and called avenues, east west streets are named and called streets | 18:33 |
| jeblair | wow, the odds of bluebox catching a 2node job are really small :) | 18:33 |
| mtreinish | I was playing with: locally: https://review.openstack.org/#/c/243866/ | 18:34 |
| jeblair | mtreinish: can you point me at the runs? | 18:34 |
| *** jaypipes has quit IRC | 18:34 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool: Write the cloud name into the provider info https://review.openstack.org/247084 | 18:34 |
| mordred | jeblair: there you go | 18:34 |
| anteaya | mtreinish: that's okay, I'll keep hoping | 18:34 |
| clarkb | anteaya: and we have 5 quandrants that divide the city and determine where you start counting numbers | 18:34 |
| anteaya | clarkb: that helps, apparently utah is a mess, everything is called temple street | 18:34 |
| mtreinish | jeblair: sure one sec, I'll have to dig it out of the db (the dashboard isn't there yet) | 18:34 |
| clarkb | (just ignore the one place where the numbers go negative) | 18:34 |
| *** zns has joined #openstack-infra | 18:34 | |
| anteaya | clarkb: ha ha ha | 18:34 |
| anteaya | the art district | 18:35 |
| clarkb | I guess you have to ignore that we have 5 "quad"rants too | 18:35 |
| anteaya | or where the artists live, not the galleries | 18:35 |
| anteaya | ha ha ha | 18:35 |
| * anteaya puts her hand over her eyes | 18:36 | |
| mordred | the streets here below 1 are letters if you're going east-west or a names if you're going north-south | 18:36 |
| mtreinish | jeblair: http://logs.openstack.org/51/165751/21/gate/gate-grenade-dsvm-multinode/8c37b8d/ and http://logs.openstack.org/86/237786/1/gate/gate-grenade-dsvm-multinode/0253513/ | 18:36 |
| zaro | clarkb: i'm testing review-dev | 18:36 |
| *** MarkAtwood has joined #openstack-infra | 18:37 | |
| mordred | of course, the area/size of city below 1st street is larger than most other cities I believe | 18:37 |
| *** tonytan4ever has quit IRC | 18:37 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/puppet-openstackid: Improved Bower support https://review.openstack.org/245394 | 18:38 |
| mordred | since that encompases all of SoHo, TriBeCa, Chinatown, Little Italy, the Lower East Side, Two Bridges, Battery Park and the Financial District | 18:38 |
| jeblair | in the civilized part of nyc (that is, not the area mordred is talking about) you can even calculate distance based on street names alone (20 street blocks/mile; ~5 avenue blocks/mile) | 18:39 |
| anteaya | mordred: they must be hard pressed to find words for names that start with a | 18:40 |
| anteaya | jeblair: I did not know that | 18:40 |
| mordred | anteaya: oh - they streets below 1st are not in alphabetical order | 18:40 |
| mordred | anteaya: they are more london-like in their sprawl | 18:40 |
| mordred | what's more - on top of what jeblair said, you can in fact determine what streets a street number falls between up there | 18:41 |
| anteaya | mordred: I thought you said below 1 going north-south all street names start with a | 18:41 |
| mordred | as street numbers radiate outward from 5th ave (center of the island) | 18:41 |
| anteaya | which would put pressure on the pool of names that start with a | 18:41 |
| jeblair | anteaya, mordred: see everything below 14 is confusing. :) | 18:42 |
| mordred | anteaya: streets below 1st ave which run north/south are named A, B, C and D | 18:42 |
| mordred | anteaya: avenues below 1st ave which run north/south are named A, B, C and D | 18:42 |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 18:42 |
| anteaya | I thought 5th ave was the centre of the universe | 18:42 |
| * anteaya recalculates | 18:42 | |
| mordred | anteaya: streets below 1st street are named | 18:42 |
| mordred | the intersection of 1st street and 1st avenue is a fun place | 18:42 |
| anteaya | I bet | 18:42 |
| mordred | as is the intersection of 1st avenue and 2nd street, and the intersection of 2nd avenue and 1st street | 18:43 |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 18:43 |
| openstackgerrit | Christopher Aedo proposed openstack-infra/puppet-apps_site: Add python WSGI server https://review.openstack.org/245315 | 18:43 |
| anteaya | ha ha ha | 18:43 |
| mordred | makes saying "take me to 1st and 2nd" to a cabbie hard | 18:43 |
| *** sputnik13 has joined #openstack-infra | 18:43 | |
| anteaya | that explains the look I see in most new yorker's eyes | 18:43 |
| anteaya | I say go to the big tree and turn left | 18:43 |
| anteaya | top of the hill | 18:43 |
| mordred | yeah | 18:43 |
| mordred | not so much with you here | 18:43 |
| anteaya | not so much | 18:43 |
| jeblair | anteaya: turn left where the tree used to be? | 18:43 |
| anteaya | that's the place | 18:44 |
| mordred | jeblair: that's in the east village | 18:44 |
| *** signed8b_ is now known as signed8bit_ZZZzz | 18:44 | |
| mordred | 8th street and 3rd ave :) | 18:44 |
| *** zz_dimtruck is now known as dimtruck | 18:44 | |
| Clint | a tree grows in east village | 18:44 |
| mordred | Clint: YOU LIE | 18:44 |
| anteaya | Clint: no | 18:44 |
| * Clint chuckles. | 18:44 | |
| anteaya | Clint: proof | 18:44 |
| mordred | actually | 18:44 |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 18:44 |
| mordred | many trees grow in the east village | 18:44 |
| mordred | it has a very high frequency of public gardens | 18:45 |
| *** bardia has joined #openstack-infra | 18:45 | |
| anteaya | really? | 18:45 |
| openstackgerrit | Merged openstack-infra/project-config: Only run gate-tempest-dsvm-multinode-live-migration on master https://review.openstack.org/246977 | 18:45 |
| mordred | oh yes | 18:45 |
| mordred | it's one of the things they managed to save when they prevented the EV from being turned into skyscrapers | 18:45 |
| anteaya | good for them | 18:45 |
| anteaya | yay! | 18:45 |
| jeblair | mtreinish: looks like a failure in tempest http://logs.openstack.org/51/165751/21/gate/gate-grenade-dsvm-multinode/8c37b8d/logs/grenade.sh.txt.gz | 18:45 |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 18:45 |
| anteaya | when I sit on a patio in the skyscraper place it feels like I am in a living room with a really tall ceiling | 18:46 |
| jeblair | mtreinish: so this might be 'bluebox performing differently enough to expose new races' but does not look like 'bluebox completely bombed in multinode setup' | 18:46 |
| *** pvaneck has joined #openstack-infra | 18:46 | |
| mordred | anteaya: http://www.communitygardensoftheeastvillage.com/ | 18:46 |
| anteaya | ohhh, proof! | 18:46 |
| mordred | anteaya: and http://evpcnyc.org/index2.html | 18:47 |
| *** signed8bit_ZZZzz is now known as signed8b_ | 18:47 | |
| *** tiswanso has quit IRC | 18:47 | |
| *** cloudtrainme has quit IRC | 18:47 | |
| anteaya | well there are addresses: http://communitygardensoftheeastvillage.com/Gardens-List.php | 18:47 |
| anteaya | now I understand what you mean about letters for names | 18:48 |
| *** tiswanso has joined #openstack-infra | 18:48 | |
| anteaya | I find it interesting they picked green maple leaves to use as markers on their site map | 18:49 |
| *** dprince has quit IRC | 18:49 | |
| * mordred used to live on 3rd between C and D | 18:49 | |
| mordred | anteaya: http://gothamist.com/2015/05/21/best_gardens_nyc.php | 18:49 |
| anteaya | maple trees are my favourite trees I think | 18:49 |
| mordred | anteaya: if you had declared a prefernce for a non-maple tree I would have worried about your canadianishness | 18:49 |
| anteaya | nice | 18:49 |
| anteaya | oh I like all trees | 18:49 |
| *** erlon has joined #openstack-infra | 18:50 | |
| anteaya | but maple trees are my family | 18:50 |
| mordred | ++ | 18:50 |
| anteaya | when I lived outside I was dreaming of a time when I had two houses | 18:50 |
| anteaya | a tree house for summer | 18:50 |
| anteaya | and a house below the earth for winter | 18:50 |
| anteaya | I would still like that | 18:50 |
| anteaya | just hard to get power to them | 18:50 |
| mordred | agree | 18:52 |
| anteaya | living outside was lovely | 18:52 |
| anteaya | the owls told me when I had 30 minutes left before the sun went down | 18:52 |
| anteaya | so I had time to clean up from my work and start a fire to make my dinner before it got dark | 18:53 |
| anteaya | owls are handy as timekeepers | 18:53 |
| *** EricGonczer_ has joined #openstack-infra | 18:53 | |
| mordred | I have never used an owl as a clock before | 18:53 |
| pleia2 | I think california is the first place I've lived where such an arrangement wouldn't result in being covered in mosquito bites :) | 18:53 |
| Clint | mordred: that surprises me | 18:53 |
| *** otter768 has joined #openstack-infra | 18:53 | |
| *** lykinsbd_ has joined #openstack-infra | 18:54 | |
| anteaya | pleia2: oh there were lots of mosquitos | 18:54 |
| *** lykinsbd_ has quit IRC | 18:54 | |
| anteaya | outside in canada == mosquitos | 18:54 |
| *** rpodolyaka1 has quit IRC | 18:54 | |
| *** salv-orl_ has joined #openstack-infra | 18:54 | |
| anteaya | mordred: I hadn't expected it but I moved into their home | 18:54 |
| *** salv-orl_ has quit IRC | 18:54 | |
| anteaya | and learned what they did | 18:54 |
| pleia2 | anteaya: thought so, the ones in Maine and upstate NY are as big as birds ;) | 18:54 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add internap to nodepool clouds.yaml https://review.openstack.org/247091 | 18:54 |
| *** lykinsbd_ has joined #openstack-infra | 18:54 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Pin jaraco requirements to working versions https://review.openstack.org/247092 | 18:55 |
| anteaya | pleia2: yup, they get large | 18:55 |
| *** tonytan4ever has joined #openstack-infra | 18:55 | |
| anteaya | owls are also absolutely silent when they fly | 18:55 |
| *** sridhar_ram has quit IRC | 18:56 | |
| anteaya | watching one fly towards me was surreal as I have never ever seen a bird fly at close range and not heard something | 18:56 |
| *** bpokorny has quit IRC | 18:56 | |
| *** bpokorny has joined #openstack-infra | 18:57 | |
| *** lykinsbd has quit IRC | 18:57 | |
| *** salv-orlando has quit IRC | 18:57 | |
| mordred | sandy enjoyed the owl cafe in tokyo a lot | 18:57 |
| *** mrc3 has left #openstack-infra | 18:57 | |
| *** shashank_hegde has joined #openstack-infra | 18:58 | |
| mordred | jeblair: I confirm from graphite that the new nodepool settings have taken hold for bluebox and ovh | 18:58 |
| jeblair | there was a good nature episode on owls and how they fly silently: http://www.pbs.org/wnet/nature/owl-power/11628/ | 18:58 |
| *** otter768 has quit IRC | 18:58 | |
| openstackgerrit | Apoorva Deshpande proposed openstack-infra/ciwatch: Consolidating database calls in db.py https://review.openstack.org/246723 | 18:59 |
| *** lykinsbd has joined #openstack-infra | 18:59 | |
| *** austin81 has joined #openstack-infra | 18:59 | |
| mordred | jeblair: I know you deleted the test instance I had in bluebox - did we go delete floating ips from there too? | 18:59 |
| anteaya | mordred: I missed the owl cafe | 18:59 |
| anteaya | jeblair: oh nice, thank you | 18:59 |
| jeblair | (the scalloped engine nacelles on the 787 are inspired by raptor feathers) | 19:00 |
| jeblair | mordred: not sure | 19:00 |
| *** lykinsbd_ has quit IRC | 19:00 | |
| *** doug-fis_ has joined #openstack-infra | 19:00 | |
| jeblair | mordred: i know i have done that cleanup a few times, i can't recall for certain whether i did so after that instance | 19:00 |
| anteaya | ha ha ha | 19:00 |
| anteaya | I wish jets could fly silently | 19:01 |
| mordred | jeblair: I'm going to doublecheck the places I might have left one | 19:01 |
| anteaya | or I could book a cot and a bunch of owls and been flow instead | 19:01 |
| sdague | jeblair / mordred - you know what's super awesome. The devstack patch with ebtables race support (default to off) | 19:01 |
| sdague | failed on ebtables race | 19:01 |
| sdague | http://logs.openstack.org/01/246501/5/check/gate-tempest-dsvm-postgres-full/1b4d7b7/logs/screen-n-cpu.txt.gz?level=TRACE#_2015-11-18_16_54_05_185 | 19:01 |
| jeblair | sdague: ha! | 19:01 |
| *** sergmelikyan has joined #openstack-infra | 19:02 | |
| sdague | so we're just going to have to recheck grind that one through | 19:02 |
| mordred | jeblair: yup there was one. I got rid of it. | 19:02 |
| * anteaya looks up if she can book owl transport on the travel agent website | 19:02 | |
| jeblair | sdague: why don't we reverse the dependency? | 19:02 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Disable InsecurePlatform Warnings https://review.openstack.org/247069 | 19:02 |
| sdague | jeblair: oh, I guess we could do that | 19:03 |
| jeblair | sdague: should be safe to land the d-g change enabling it first, right? we still get the same testing | 19:03 |
| sdague | yep | 19:03 |
| sdague | given it's got to recheck anyway, sure | 19:03 |
| *** signed8bit has joined #openstack-infra | 19:03 | |
| sdague | the other might race and fail on the same thigs | 19:03 |
| jeblair | yup | 19:03 |
| *** doug-fi__ has joined #openstack-infra | 19:03 | |
| sdague | so honestly, we might as well keep the order and just grind on this one | 19:03 |
| sdague | it's the same fail rate that everyone is dealing with | 19:03 |
| *** signed8b_ has quit IRC | 19:04 | |
| anteaya | oh yes, and owls do fly very slow, the wing beats looked like it was in stop time, I was amazed it could stay in the air | 19:04 |
| *** doug-fish has quit IRC | 19:04 | |
| clarkb | mmedvede: anteaya ruagair so is now still good (I know ruagair didn't get back and said today was good so I can run through things again tomorrow same time) | 19:04 |
| jeblair | sdague: true enough | 19:04 |
| anteaya | clarkb: good to go | 19:04 |
| *** doug-fi__ has quit IRC | 19:04 | |
| *** Guest12059 is now known as med_ | 19:04 | |
| *** med_ has quit IRC | 19:04 | |
| *** med_ has joined #openstack-infra | 19:04 | |
| *** doug-fish has joined #openstack-infra | 19:04 | |
| clarkb | mmedvede: anteaya ruagair the first things you will want to do is clone openstack-infra/puppet-log_processor and system-config | 19:04 |
| *** doug-fish has quit IRC | 19:04 | |
| sdague | the d-g devstack logs look like it did the right thing | 19:05 |
| anteaya | clarkb: will do | 19:05 |
| clarkb | mmedvede: anteaya ruagair those repos have the configuration and scripts in them | 19:05 |
| sdague | as there is a specific message when the tool installs the workaround which is there | 19:05 |
| *** doug-fis_ has quit IRC | 19:05 | |
| clarkb | mmedvede: anteaya ruagair then you will want to make a virtualenv and pip install pyyaml and gear | 19:05 |
| *** doug-fish has joined #openstack-infra | 19:05 | |
| *** harlowja has quit IRC | 19:05 | |
| *** ivar-lazzaro has joined #openstack-infra | 19:05 | |
| clarkb | let me know when you have that done | 19:05 |
| *** nadya_ has quit IRC | 19:06 | |
| *** ivar-lazzaro has quit IRC | 19:06 | |
| *** doug-fish has quit IRC | 19:06 | |
| anteaya | will do | 19:07 |
| mmedvede | clarkb: done | 19:07 |
| *** doug-fish has joined #openstack-infra | 19:07 | |
| clarkb | oh the other thing we will need to download is https://download.elastic.co/logstash/logstash/logstash-2.0.0.tar.gz and untar that (this is how we "install" logstash) | 19:08 |
| *** harlowja has joined #openstack-infra | 19:08 | |
| anteaya | clarkb: download and untar in the venv? or does it matter? | 19:08 |
| clarkb | anteaya: it doesn't matter I would do it next to the venv | 19:09 |
| *** dprince has joined #openstack-infra | 19:09 | |
| anteaya | okay | 19:09 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Ignore broken providers https://review.openstack.org/244340 | 19:09 |
| anteaya | pyyaml and gear installed in venv, gear threw a ton of warnings | 19:09 |
| jeblair | mordred, mgagne: are we waiting on activation or is that done? | 19:10 |
| mordred | jeblair: we're enabled | 19:10 |
| mordred | jeblair: lemme check on the nj activation | 19:10 |
| *** ivar-lazzaro has joined #openstack-infra | 19:10 | |
| mgagne | jeblair it's done. I'm waiting for details around which region should be used so you don't have to move around if it's the wrong one | 19:10 |
| *** sabeen1 has joined #openstack-infra | 19:11 | |
| jeblair | cool, i just tried to create a host in nyj01 and got | 19:11 |
| jeblair | | fault | {"message": "No valid host was found. Exceeded max scheduling attempts 3 for instance 283c1c41-9d02-4e06-b85e-93343dc09364. Last exception: [u'Traceback (most recent call last):\ | | 19:11 |
| jeblair | | | ', u' File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2229, in _do", "code": 500, "created": "2015-11-18T19:09:21Z"} | | 19:11 |
| mordred | two things | 19:11 |
| clarkb | mmedvede: anteaya let me know when that download is done (its a bit big so may take some time) | 19:12 |
| mordred | a) mgagne sorry, I did not see that you were still waiting on region choice, I have activated nyj01 in the openstackjenkins account | 19:12 |
| mordred | b) jeblair I have not activated any regions in opentsackinfra yet | 19:12 |
| anteaya | clarkb: downloaded | 19:12 |
| mmedvede | clarkb: done | 19:12 |
| jeblair | mordred: ah, that is the one i was launching in :) | 19:12 |
| mordred | so I can either activate nyj since we have that active in the other account, or I can wait for mgagne to get verification that that's a good region | 19:12 |
| mgagne | mordred no problem, I just wish to make those details clear for everyone so there is no misunderstanding | 19:12 |
| mordred | ++ | 19:13 |
| clarkb | one sec salami made a mess | 19:13 |
| anteaya | clarkb: unpacked | 19:13 |
| anteaya | clarkb: k | 19:13 |
| anteaya | that salami | 19:13 |
| clarkb | its tasty but covered in pepper that goes everyhwere | 19:14 |
| *** sabeen3 has joined #openstack-infra | 19:14 | |
| clarkb | we are going to work our way backwards up the stack this means start logstash first then the things that feed into it | 19:14 |
| *** dprince has quit IRC | 19:14 | |
| anteaya | k | 19:14 |
| *** hdd has joined #openstack-infra | 19:14 | |
| mmedvede | clarkb: dumb question, I should be probably running it in a VM? | 19:14 |
| anteaya | ah yeah, memories from when I ate meat | 19:14 |
| jeblair | mgagne: should we proceed with nyj01 (even if we might have to move) so we can continue set-up, or do you think you'll have an answer on a region soon and we should just go make a sandwich? :) | 19:15 |
| clarkb | mmedvede: I don't everything runs in foreground happily without affecting my OS | 19:15 |
| clarkb | mmedvede: I suppose if you are paranoid you can | 19:15 |
| mordred | jeblair: you know you have to specify net-id for boot, yeah? | 19:15 |
| mgagne | jeblair up to you, I just wish to avoid you wasting time and money =) | 19:15 |
| *** MarkAtwo_ has joined #openstack-infra | 19:15 | |
| anteaya | mmedvede: I'm not, your call | 19:15 |
| jeblair | mordred: nope! | 19:15 |
| bardia | asselin: I think there is a bug in the puppet scripts that sets up zuul. It writes the gerrit ssh key to /var/lib/zuul/ssh/id_rsa whereas it should be writing it to /home/zuul/.ssh/id_rsa | 19:15 |
| clarkb | we need to make a small change to logstash's config before starting it (tell it to write to stdout instead of ES) | 19:15 |
| mgagne | as billing details still need to be adjusted | 19:15 |
| mmedvede | clarkb: if it works on yours, I see no reason then | 19:15 |
| clarkb | that file is at system-cofngi/modules/openstack_project/templates/logstash/indexer.yaml.erb | 19:15 |
| *** dprince has joined #openstack-infra | 19:15 | |
| bardia | asselin: that was the bug that was failing my setup to start builds based on changes upstream | 19:15 |
| *** MarkAtwood has quit IRC | 19:16 | |
| *** sabeen1 has quit IRC | 19:16 | |
| mordred | jeblair: you will need to pass nics=[{'net-id': $netid}] to the boot call - because there will be two networks in that account | 19:16 |
| clarkb | oh its indexer.conf.erb | 19:16 |
| mgagne | mordred this makes me think I need to set this router:external prop | 19:16 |
| jeblair | mordred: launch-node has a --net-label argument | 19:16 |
| mordred | cool. that will work | 19:16 |
| jeblair | mgagne: ++ :) | 19:16 |
| mordred | mgagne: yes - that will be helpful for all of your users | 19:16 |
| clarkb | at the end of that file is the output section we need to comment out or remove the elasticsearch block and replace it with stdout {} | 19:16 |
| openstackgerrit | Komei Shimamura proposed openstack-infra/project-config: Add devstack-plugin-kafka project https://review.openstack.org/247094 | 19:17 |
| mordred | jeblair: --net-label will work fine | 19:17 |
| mgagne | no argument here | 19:17 |
| mordred | jeblair: you wnat the network with wan in the name | 19:17 |
| jeblair | mordred: want to enable nyj01 for infra? | 19:17 |
| bardia | asselin: zuul-merger was failing to clone the repo from review.openstack.org because of misplaced ssh_key | 19:17 |
| *** moravec has quit IRC | 19:17 | |
| mordred | jeblair: working on it | 19:18 |
| mgagne | mordred hold on, I see programmatic errors on our sie | 19:18 |
| mgagne | side* | 19:18 |
| Shrews | SpamapS: Can you get time today to review https://review.openstack.org/246963 for us? Turns out an ansible module was released months back that depends on that API. :/ | 19:18 |
| *** dramalho has joined #openstack-infra | 19:18 | |
| clarkb | anteaya: mmedvede once thats done save then start logstsah (this will foreground) with `logstash-2.0.0/bin/logstash -f path_to_config -w 1` | 19:18 |
| *** dprince has quit IRC | 19:19 | |
| anteaya | http://paste.openstack.org/show/479320/ | 19:19 |
| *** mriedem has quit IRC | 19:20 | |
| jeblair | mordred: is a neutron network label the same thing as a name? | 19:20 |
| mordred | yes | 19:20 |
| jeblair | *headdesk* | 19:20 |
| jeblair | why couldn't we have just called it a project? ;) | 19:21 |
| *** cloudtrainme has joined #openstack-infra | 19:21 | |
| clarkb | anteaya: keep the output{} and put stdout{} in it | 19:21 |
| mmedvede | anteaya: I think you need it wrapped, { stdout {} } | 19:21 |
| clarkb | mmedvede: yup | 19:21 |
| anteaya | clarkb: oh | 19:21 |
| mmedvede | clarkb: \o/, got "Logstash startup completed" | 19:21 |
| clarkb | the -w 1 flag is important because we use the multiline filter which can't run with multiple threads | 19:22 |
| clarkb | -w 1 says run a single thread (default is num_cpus/2) | 19:22 |
| *** doug-fish has quit IRC | 19:22 | |
| mordred | jeblair: I honestly don't know what net.label is | 19:22 |
| *** dramalho has quit IRC | 19:22 | |
| mordred | jeblair: the attribute of the neutron object is name | 19:22 |
| clarkb | anteaya: let us know when you have that running | 19:22 |
| mordred | RESP BODY: {"network": {"status": "ACTIVE", "subnets": ["e9440419-d276-4c3d-9c18-7991b77e2465", "ef3bde66-8ac4-4356-8ab4-099519ba218a"], "name": "Ext-Net", "router:external": true, "tenant_id": "34042578293510", "admin_state_up": true, "shared": false, "port_security_enabled": true, "id": "7da74520-9d5e-427b-a508-213c84e69616"}} | 19:23 |
| anteaya | http://paste.openstack.org/show/479321/ | 19:23 |
| anteaya | clarkb: okay don't let me slow you down | 19:23 |
| *** doug-fish has joined #openstack-infra | 19:23 | |
| *** bapalm has quit IRC | 19:23 | |
| mmedvede | is paste.o.o just slow for me? | 19:23 |
| anteaya | mmedvede: no it is slow | 19:24 |
| *** elemoine has joined #openstack-infra | 19:24 | |
| *** tdurakov_ has quit IRC | 19:24 | |
| anteaya | mmedvede: if I'm wrong again, can you paste what you have that works as a reply? | 19:24 |
| mordred | jeblair: and I don't see anywhere in python-neutronclient that associates the word label with name. nor is it listed as an optional component of the json return structure | 19:24 |
| mordred | jeblair: OH! | 19:25 |
| mmedvede | anteaya: http://paste.openstack.org/show/479322/ | 19:25 |
| *** dkranz has quit IRC | 19:25 | |
| openstackgerrit | Komei Shimamura proposed openstack-infra/project-config: Add devstack-plugin-kafka project https://review.openstack.org/247094 | 19:25 |
| *** tdurakov_ has joined #openstack-infra | 19:25 | |
| *** bapalm has joined #openstack-infra | 19:26 | |
| anteaya | mmedvede: thank you, I will copy that | 19:26 |
| clarkb | its probably worth addressing why we don't also run an elasticsearch. elasticsearch has basically ust worked and when debugging/testing its just easier to ignore it becuase the problems typically are in the parsing of the events | 19:27 |
| *** dprince has joined #openstack-infra | 19:27 | |
| *** trown|lunch is now known as trown | 19:28 | |
| mordred | jeblair: that is a call in launc_node to nova_client.networks.list() | 19:28 |
| anteaya | okay fair enough | 19:28 |
| *** tiswanso has quit IRC | 19:28 | |
| anteaya | clarkb: mmedvede I'm now caught up: http://paste.openstack.org/show/479323/ | 19:28 |
| *** tiswanso has joined #openstack-infra | 19:29 | |
| jeblair | mordred: so novaclient calls something a label | 19:29 |
| *** tiswanso has quit IRC | 19:29 | |
| mordred | jeblair: right. because nova-network | 19:29 |
| *** tiswanso has joined #openstack-infra | 19:30 | |
| *** mriedem has joined #openstack-infra | 19:30 | |
| *** mahatic has quit IRC | 19:30 | |
| jeblair | oh i see 3 networks there now, 2 WAN, one with a cidr. i'm guessing that's the one. | 19:30 |
| clarkb | mmedvede: anteaya great so logstash is running and ready to parse events based on that config. Next step is to give it events to parse. We do that with a gearman client worker setup | 19:30 |
| mordred | yah | 19:30 |
| mordred | jeblair: also, I checked, internap does the nova network proxy | 19:31 |
| mgagne | mordred we do? | 19:31 |
| mordred | so the networks.list() will work and label will match the name | 19:31 |
| clarkb | mmedvede: anteaya: to set that up source your virtualenv with gear and pyyaml in it and run geard -d (this runs a gearman server in the foreground) | 19:31 |
| mordred | mgagne: yes | 19:31 |
| mordred | >>> c.nova_client.networks.list() | 19:31 |
| mordred | [<Network: inap-17037-WAN1654>, <Network: inap-17037-LAN3631>] | 19:31 |
| mgagne | oh yea | 19:31 |
| mordred | HP, fwiw, does not | 19:31 |
| clarkb | mmedvede: anteaya: then copy http://paste.openstack.org/show/479324/ and save it as worker.yaml (this is our worker process config that tells it which gearman to connect to and where to talk to logstash at) | 19:31 |
| mordred | so using HP to verify this will not go well :) | 19:31 |
| anteaya | clarkb: oh sorry I was running logstash in the venv, I will stop it and run it not in the venv | 19:32 |
| clarkb | anteaya: thats fine | 19:32 |
| clarkb | anteaya: the venv stuff should only affect ypthon which logstash isn't | 19:32 |
| anteaya | okay | 19:32 |
| mordred | jeblair: you should be active in nyj now - so it should work | 19:33 |
| clarkb | once you have that you can start a geraman worker (again with the venv) using `python puppet-log_processor/files/log-gearman-worker.py -c path_to_worker.yaml --foreground -d path_to_logfile_worker.log` | 19:33 |
| jeblair | mordred: is there a shade launch-node? | 19:34 |
| mordred | jeblair: you wrote one | 19:34 |
| jeblair | ah here it is https://review.openstack.org/234975 | 19:35 |
| jeblair | oh pep8 | 19:35 |
| SpamapS | Shrews: LGTM | 19:35 |
| jeblair | that's why we can't have nice things | 19:35 |
| mordred | jeblair: I need to make you a follow up patch for that | 19:36 |
| mordred | one sec | 19:36 |
| mmedvede | clarkb: got a bunch of gear connections show up on geard, and logfile worker log shows "Polling 1 connections" | 19:36 |
| *** sergmelikyan has quit IRC | 19:36 | |
| *** lykinsbd has quit IRC | 19:36 | |
| clarkb | mmedvede: aewsome that is what I expect | 19:37 |
| clarkb | the last step is to get a gearman client that tells the worker to do work. Instead of using the one in puppet-log_processor I tend to use something simpler like http://paste.openstack.org/show/479325/ | 19:37 |
| *** nadya has joined #openstack-infra | 19:37 | |
| clarkb | this way I can more easily send very specific events rather than needing to involve jenkins. If you grab the code there and save it in a file we can run it in the venv too in order to trigger a job that will parse that console log with logstash | 19:38 |
| clarkb | when you do this you should see a bunch of output from logstash with the parsed output | 19:38 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add a shade launch-node https://review.openstack.org/234975 | 19:38 |
| *** Sukhdev has joined #openstack-infra | 19:38 | |
| anteaya | http://paste.openstack.org/show/479328/ | 19:39 |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-nodepool: Allow a builder-worker to run on the nodepool host https://review.openstack.org/240093 | 19:40 |
| anteaya | clarkb: gearman_client.py? | 19:41 |
| clarkb | anteaya: that looks good, the GRAB_JOB message i sthe worker saying hey give me owrk but there isn't work to do until you send it some jobs | 19:41 |
| clarkb | anteaya: ya you cna call it what you want and just python gearman_client.py it | 19:41 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Use shade for net-id https://review.openstack.org/247099 | 19:41 |
| clarkb | it takes no args bceause its a simple test script | 19:41 |
| mordred | jeblair: ^^ that does the net-id thing from shade | 19:41 |
| jeblair | mordred: oh, heh, i was just writing that :) | 19:41 |
| jeblair | mordred: you didn't tell me what your followup patch was :) | 19:41 |
| mordred | mgagne: any chance you can flip the router:external bit on these networks? | 19:42 |
| mordred | jeblair: heh. well, if you'd like to write it - that's cool :) | 19:42 |
| mgagne | oh, I forgot, sorry! doing it now | 19:42 |
| mordred | no worries at all! | 19:42 |
| jeblair | mordred: nope, i had just decided to type approximately those characters, so cool. :) | 19:42 |
| clarkb | mmedvede: anteaya once you get that done with output from logstash let me know and we can run through everything that happens (and get a bit more indepth with the logstash config) | 19:43 |
| mordred | jeblair: neat | 19:43 |
| clarkb | but I think its easier to understand with it actually doing something first rather than me hand waving | 19:43 |
| mmedvede | clarkb: yes, that worked, got a lot of output from logstash | 19:43 |
| mgagne | done | 19:44 |
| *** fitoduarte has quit IRC | 19:44 | |
| mordred | jeblair: so, my quick and dirty script from last week had a sad with internap - but I have tentatively chalked that up to me working too quick - so if you have trouble with the boot, I'd love to jump on and debug | 19:44 |
| *** fitoduarte has joined #openstack-infra | 19:44 | |
| jeblair | mordred: but without the extra ')' | 19:44 |
| anteaya | http://paste.openstack.org/show/479330/ | 19:44 |
| mgagne | mordred: anything we can do? | 19:44 |
| mordred | because it's _possible_ that the error I was seeing wasn't pebkac | 19:44 |
| mordred | mgagne: duno if there is a real problem yet | 19:44 |
| *** ddieterly has quit IRC | 19:45 | |
| mordred | mgagne: and if there is, it's likely on my side | 19:45 |
| *** MarkAtwo_ is now known as MarkAtwood | 19:45 | |
| smarcet1981 | mordred: hi, could u review https://review.openstack.org/#/c/245394/ please ? :) | 19:45 |
| mgagne | mordred: ok, as stated, I hot fixed a bug on our side around network metadata generation | 19:45 |
| anteaya | clarkb: yes I learn best my doing something, even if I don't know the significane of what I am doing the first time | 19:45 |
| anteaya | clarkb: feel free to make notes in my paste if you want | 19:45 |
| *** ayoung has joined #openstack-infra | 19:45 | |
| jeblair | mordred: have you made a clouds.yaml for puppetmaster yet? | 19:46 |
| clarkb | anteaya: so that ran look at the terminal where logstash is running | 19:46 |
| mordred | jeblair: I have not - would you like me to? | 19:46 |
| clarkb | anteaya: you should have tons of output | 19:46 |
| anteaya | clarkb: as that is my way of both communicating my status to you and keeping notes for later | 19:46 |
| *** gildub_ has joined #openstack-infra | 19:46 | |
| jeblair | mordred: yeah, i guess we need to translate the whole ~ci-launch directory into clouds.yaml | 19:46 |
| mordred | yah | 19:47 |
| anteaya | clarkb: I don't http://paste.openstack.org/show/479331/ | 19:47 |
| mordred | jeblair: I could do that by hand for the moment and follow it up with a puppet change | 19:47 |
| *** pradk_ has quit IRC | 19:47 | |
| jeblair | mordred: by hand for now sounds good... there's a lot of stuff not in hiera, so puppet will be some work. | 19:47 |
| clarkb | anteaya: is the worker still running? | 19:47 |
| mordred | ++ | 19:48 |
| anteaya | clarkb: my log process got control-c'd sorry about that | 19:48 |
| *** tsg has quit IRC | 19:48 | |
| clarkb | anteaya: everything needs to be running will need 4 terminals (or screen windows) | 19:48 |
| *** hdd has quit IRC | 19:48 | |
| anteaya | clarkb: yup, now I have output | 19:48 |
| anteaya | I think when I made a new tab I must have control-c'd | 19:48 |
| *** dkranz has joined #openstack-infra | 19:48 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Use shade for net-id https://review.openstack.org/247099 | 19:49 |
| *** lykinsbd has joined #openstack-infra | 19:49 | |
| *** Sukhdev has quit IRC | 19:49 | |
| anteaya | http://paste.openstack.org/show/479332/ | 19:50 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Add documentation for third-party-ci setup https://review.openstack.org/227584 | 19:50 |
| clarkb | anteaya: mmedvede: ok great so now ou have functioning log worker and logstash processes | 19:50 |
| *** placko_ has joined #openstack-infra | 19:50 | |
| anteaya | yay | 19:50 |
| openstackgerrit | Yan Ning proposed openstack-infra/project-config: add seperate builder/job for monasca plugin in devstack install https://review.openstack.org/244893 | 19:50 |
| *** onder has quit IRC | 19:50 | |
| *** kushal has quit IRC | 19:50 | |
| clarkb | anteaya: mmedvede the basic flow here is the client says "hey here is a log file with some information about it please run it through the push-log gearman function | 19:51 |
| clarkb | anteaya: mmedvede the log worker has registered itself as handling the push-log gearman function so the geard server tells it to wake up and perform that job given the log file and info about it | 19:52 |
| clarkb | anteaya: mmedvede the log worker then fetches the log file from https://logs.openstack.org/, unzips it if necessary, then sends a json event per line to logstash over tcp port 9999 | 19:52 |
| Swanson | zuul question. What is the minimum I have to setup for smtp to actually function? | 19:53 |
| *** lykinsbd has quit IRC | 19:53 | |
| *** onder has joined #openstack-infra | 19:53 | |
| clarkb | anteaya: mmedvede each event contains a bunch of info, the current timestamp, the message itself (which usually contains another timestamp), jenkins info like was the job a success, is the job voting, etc and so on | 19:53 |
| clarkb | anteaya: mmedvede if you want to see that you can `sudo tcpdump -A -i any port 9999` then run the client script again | 19:54 |
| *** jed56 has quit IRC | 19:54 | |
| *** MarkAtwood has quit IRC | 19:54 | |
| clarkb | anteaya: mmedvede: one of the most important pieces of info in those events for us is the tags field. These tags contain info about the file like its filename and other log processing attributes which we use in the logstash config to determine how to parse the file | 19:55 |
| *** rpodolyaka1 has joined #openstack-infra | 19:55 | |
| clarkb | anteaya: mmedvede some of the common tags are syslog, console, oslofmt, apachefmt, etc basically "what kind of log this is" | 19:55 |
| clarkb | anteaya: mmedvede in this case console.html is in our tags list because that is the basename of the url which means we hit https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/logstash/indexer.conf.erb#n15 and enter that block for parsing console logs | 19:56 |
| openstackgerrit | Komei Shimamura proposed openstack-infra/project-config: Add devstack-plugin-kafka project https://review.openstack.org/247094 | 19:57 |
| jroll | someone mind landing ironic's reno job? yolanda you had a +2 previously https://review.openstack.org/#/c/242152/ | 19:57 |
| clarkb | anteaya: mmedvede within that block we have two filters, the first handles multiline events. The short version of that multiline filter is if I have no timestamp on this message then it is a part of the previous message | 19:59 |
| clarkb | anteaya: mmedvede when the next event to not match that filter comes in the previous multiline event is considered done and is sent to the output | 19:59 |
| anteaya | clarkb mmedvede I haven't done a lot with tcpdump before I keep getting tcpdump: Invalid adapter index, I suspect I am not selecting a good port | 19:59 |
| clarkb | anteaya: port 9999 is just a match rule it may not like -i any. You can change that to -i lo I think | 20:00 |
| clarkb | which says tcpdump on loopback interface | 20:00 |
| anteaya | I will try | 20:00 |
| clarkb | and the -A says print all the datas | 20:00 |
| anteaya | my fault I think I had read any as <any> and was trying numbers | 20:01 |
| anteaya | sudo tcpdump -A -i any 9999 | 20:01 |
| anteaya | tcpdump: syntax error | 20:01 |
| clarkb | anteaya: port 9999 | 20:01 |
| clarkb | that tells tcpdump to only dump packets that were sent to or from port 9999 | 20:01 |
| *** Rockyg has joined #openstack-infra | 20:01 | |
| anteaya | thanks | 20:02 |
| *** zns has quit IRC | 20:02 | |
| anteaya | listening now, no output | 20:02 |
| anteaya | something now | 20:02 |
| *** lykinsbd has joined #openstack-infra | 20:02 | |
| clarkb | anteaya: you will have to run your client script again to trigger communication over that port | 20:03 |
| anteaya | yes I do that | 20:03 |
| clarkb | anteaya: and tcpdump should print out a bunch of json in resposne? | 20:04 |
| anteaya | and then logstash finishes with these two lines: http://paste.openstack.org/show/479334/ | 20:04 |
| *** ZZelle_ has joined #openstack-infra | 20:04 | |
| anteaya | it could be json | 20:04 |
| jroll | tristanC: ttx: if either of you are around, mind PMing me please? | 20:05 |
| anteaya | yes running the client script produces output in the logstash terminal and the tcpdump terminal | 20:05 |
| *** Sukhdev has joined #openstack-infra | 20:05 | |
| clarkb | anteaya: the tcpdump terminal shows the communication format between the gearman worker and logstash | 20:06 |
| clarkb | the logstash terminal is the result of logstash parsing that input | 20:06 |
| anteaya | well that is what I am confused by | 20:06 |
| anteaya | as the output in the logstash terminal doesn't look like parsed anything to me | 20:07 |
| anteaya | it just looks like a build log | 20:07 |
| anteaya | or is that what it is parsing? | 20:07 |
| clarkb | anteaya: its the parsed timestamp which should be from when the job ran a few days ago and not now, the source of the message (127.0.0.1) and finally the message itself | 20:08 |
| mordred | jeblair: ok. clouds.yaml exists | 20:08 |
| anteaya | that is a parsed build log? http://paste.openstack.org/show/479335/ | 20:08 |
| jeblair | mordred: do we need project name and project id? | 20:09 |
| anteaya | okay thanks yup just me being slow | 20:09 |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: Update bashate to 0.3.2 https://review.openstack.org/236815 | 20:09 |
| asselin | bardia, that is the correct location | 20:09 |
| clarkb | anteaya: ya, by default stdout is only going to show you those three fields | 20:10 |
| jeblair | mordred: i only did name in my system-config change | 20:10 |
| anteaya | clarkb: okay | 20:10 |
| openstackgerrit | Merged openstack-infra/shade: Add user group assignment API https://review.openstack.org/246963 | 20:10 |
| openstackgerrit | Yan Ning proposed openstack-infra/project-config: add seperate builder/job for monasca plugin in devstack install https://review.openstack.org/244893 | 20:10 |
| clarkb | anteaya: if you want to see all of the info that logstash knows about you can change stdout {} to stdout{ codec => json_lines {} } and restart that process | 20:10 |
| *** MarkAtwood has joined #openstack-infra | 20:11 | |
| clarkb | that should print the complete parsed events on the other end | 20:11 |
| * anteaya does so | 20:11 | |
| *** salv-orlando has joined #openstack-infra | 20:11 | |
| clarkb | for console logs though really all we are parsing is the timestamp and message | 20:11 |
| clarkb | since there isn't much to them | 20:11 |
| anteaya | right just trying to understand the things I currently have running | 20:11 |
| mordred | jeblair: we do not - I just had them so I put both there | 20:11 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Add test for os_user_group Ansible module https://review.openstack.org/247105 | 20:11 |
| *** jtomasek has quit IRC | 20:12 | |
| mtreinish | jeblair: yeah it doesn't look like the setup is inherently broken on bluebox | 20:12 |
| clarkb | anteaya: note that the worker should detect the disconnect from the logstash process then wait like a minute before reconnecting iirc | 20:12 |
| mtreinish | jeblair: although the other failure looks like it was caused by io errors on the disk | 20:12 |
| clarkb | anteaya: so there may be a delay after you run the client | 20:12 |
| mordred | jeblair: we had a spike of node create failure son bb | 20:12 |
| *** ningy__ has joined #openstack-infra | 20:13 | |
| *** ddieterly has joined #openstack-infra | 20:13 | |
| *** e0ne has quit IRC | 20:13 | |
| anteaya | http://paste.openstack.org/show/479337/ line 596 | 20:14 |
| anteaya | clarkb: okay | 20:14 |
| *** ningy__ has quit IRC | 20:14 | |
| jeblair | mordred: neat; i need to lunch; if you don't want to look into bb i can when i get back | 20:14 |
| *** ldnunes has quit IRC | 20:14 | |
| jeblair | mordred: shade-launch-node is building an instance on internap now | 20:14 |
| clarkb | anteaya: ya that config looks right | 20:15 |
| clarkb | anteaya: lgostash should yell if it isn't | 20:15 |
| mordred | jeblair: cool | 20:15 |
| clarkb | and we can go from there to debug | 20:15 |
| anteaya | clarkb: okay it is delaying as you said | 20:16 |
| anteaya | thinking about making a tea while it makes up its mind | 20:16 |
| clarkb | anteaya: that delay is there because logstash is slow to start so if it restarts for whatever reason we want to make sure we wait long enough for it to be listening again before talking to it | 20:16 |
| mordred | 2015-11-18 20:16:02,639 DEBUG nodepool.NodePool: Instance devstack-trusty-bluebox-sjc1-6323 (f0aa4938-2fa1-4fbf-9ac8-3508fea6976b) in bluebox-sjc1 has no nodepool metadata | 20:16 |
| anteaya | right | 20:16 |
| mordred | that's fun | 20:16 |
| anteaya | yeah everything else is running | 20:16 |
| * anteaya waits | 20:16 | |
| anteaya | there it goes | 20:17 |
| *** bhunter71 has quit IRC | 20:17 | |
| jeblair | mordred: that may be a leak from my local testing | 20:17 |
| mordred | ah. k | 20:17 |
| anteaya | clarkb mmedvede line 609: http://paste.openstack.org/show/479338/ | 20:18 |
| jeblair | mordred: safe to delete if you want | 20:18 |
| *** MarkAtwood has quit IRC | 20:18 | |
| anteaya | clarkb: that was more along the lines of what my eyes were expecting I think | 20:18 |
| *** dramalho has joined #openstack-infra | 20:19 | |
| clarkb | anteaya: cool | 20:19 |
| jeblair | mordred: (my local nodepool has launched a lot of instances!) | 20:19 |
| * mmedvede changing settings also | 20:19 | |
| anteaya | clarkb: wow build_status and tags for every line | 20:19 |
| anteaya | mmedvede: cool | 20:19 |
| clarkb | anteaya: mmedvede so now the next steps for testing logstash 2.0 are to edit the client to grab a syslog file (and set the appropriate tags for that) and make sure syslog goes through fine | 20:19 |
| clarkb | anteaya: mmedvede and oslofmt and swift and apache etc | 20:19 |
| clarkb | anteaya: yes because we are line based every line has a complete picture of the job as a whole | 20:20 |
| clarkb | anteaya: its inefficient | 20:20 |
| anteaya | yes | 20:20 |
| anteaya | now I have a better understanding why it takes 16 or so workers and hours of delay | 20:21 |
| *** gondoi has quit IRC | 20:21 | |
| *** moravec has joined #openstack-infra | 20:21 | |
| clarkb | anteaya: thats actually more because our logs are hard to parse and can be very large than the metadata | 20:21 |
| clarkb | anteaya: but ya | 20:21 |
| anteaya | so we have been parsing the same gate infra pupppet apply trusty job log | 20:21 |
| *** yamahata has quit IRC | 20:21 | |
| anteaya | I had just copy/pasted the client code | 20:22 |
| clarkb | anteaya: yes, you can change the client to do other files (thats how I test) you just have to also edit the tags to have the correct tags | 20:22 |
| clarkb | so for a nova log file it would be oslofmt | 20:22 |
| clarkb | for a syslog file it would be syslog | 20:22 |
| clarkb | and so on | 20:22 |
| clarkb | and if you read the logstash indexer.conf file you can see what the switches are | 20:22 |
| * anteaya finds the indexer.conf file | 20:23 | |
| clarkb | anteaya: the one you edited for json_lines | 20:23 |
| *** dramalho has quit IRC | 20:23 | |
| anteaya | there it is | 20:24 |
| clarkb | https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/logstash/indexer.conf.erb#n15 is the one we currently use | 20:24 |
| clarkb | but if you switch to a nova log file and change the tag to oslofmt you will hit https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/logstash/indexer.conf.erb#n31 instead | 20:25 |
| *** ParsectiX has quit IRC | 20:25 | |
| * anteaya tries that | 20:26 | |
| clarkb | also I think you may need to remove the .gz from the loglines as the workers are smart about adding them | 20:26 |
| clarkb | (and will do the wrong thing if it is already there) | 20:26 |
| clarkb | s/loglines/log file urls/ | 20:26 |
| *** kozhukalov_ has quit IRC | 20:27 | |
| clarkb | anteaya: mmedvede but ya the process to go through now is grab the various cases and make sure they appear to parse ok. one thing to look for in the output of json_lines is the grokparsefailure tag | 20:30 |
| clarkb | anteaya: mmedvede that is an indication that logstash failed to handle something in the logfile | 20:30 |
| clarkb | anteaya: mmedvede if you end up running through those maybe ping back here with results (and questions if they come up) | 20:31 |
| anteaya | clarkb: http://paste.openstack.org/show/479339/ line 619 | 20:32 |
| *** dkranz has quit IRC | 20:32 | |
| *** dprince has quit IRC | 20:32 | |
| anteaya | wonderful thank you | 20:32 |
| anteaya | clarkb: have we gotten to a place where you feel pleased that you got us here? | 20:32 |
| anteaya | are we useful yet or just one step closer? | 20:33 |
| clarkb | anteaya: thats still a console log not an oslofmt | 20:33 |
| clarkb | anteaya: so for that you cna change the url but the tags should remain the way it was | 20:33 |
| clarkb | anteaya: when I say nova log I mean screen-n-cpu.txt and friends | 20:33 |
| clarkb | (logs written by nova) | 20:33 |
| anteaya | I don't know what an oslofmt is | 20:33 |
| anteaya | ah I will try that | 20:33 |
| clarkb | its the tag we use to denote that a logfile is in the format that oslo writes | 20:34 |
| clarkb | which is different than the format console logs are written in anddifferent than syslog | 20:34 |
| anteaya | oh okay yes | 20:34 |
| mordred | jeblair: when you get back, I do not see failures in the nodepool log to correspond to the error entries in graphite | 20:34 |
| anteaya | logs from screen process == oslofmt | 20:34 |
| clarkb | anteaya: logs from oslo :) | 20:35 |
| clarkb | anteaya: some of the screen logs (like swift) are in other formats | 20:35 |
| clarkb | oh though if it is screen you should have something like tags = ["screen", "oslofmt"] | 20:35 |
| clarkb | because we actually need to remove the screen header junk at the start | 20:35 |
| *** bhunter71 has joined #openstack-infra | 20:36 | |
| *** harlowja has quit IRC | 20:36 | |
| *** nadya has quit IRC | 20:36 | |
| openstackgerrit | Merged openstack/requirements: Updated oslo.db to 3.2.0 https://review.openstack.org/246325 | 20:36 |
| anteaya | clarkb: oh okay | 20:37 |
| anteaya | http://paste.openstack.org/show/479341/ line 662 | 20:37 |
| anteaya | clarkb: I will add that | 20:38 |
| *** tongli has quit IRC | 20:38 | |
| clarkb | anteaya: ya that looks better | 20:39 |
| pabelanger | So, just learned something today: http://governance.openstack.org/reference/project-testing-interface.html | 20:39 |
| *** spzala has joined #openstack-infra | 20:40 | |
| openstackgerrit | Merged openstack/requirements: Bump oslo.utils to 2.8.0 https://review.openstack.org/246149 | 20:40 |
| pabelanger | what is the history of calling tox -evenv python setup.py build_sphinx over tox -edocs ? | 20:40 |
| anteaya | http://paste.openstack.org/show/479342/ line 675 | 20:40 |
| clarkb | pabelanger: tox -edocs did not exist and is redundant | 20:40 |
| *** hashar has joined #openstack-infra | 20:40 | |
| greghaynes | mordred: SpamapS https://review.openstack.org/#/c/247116/ | 20:40 |
| clarkb | pabelanger: so the history is "we already have a taox target for arbitrary commands its call venv" | 20:40 |
| clarkb | pabelanger: then after some time people decided that remembering how to make a sphinx build was hard and we should have docs targets | 20:41 |
| clarkb | pabelanger: but ther ewas no standardization so we eneded up with like 3 different targets | 20:41 |
| *** harlowja has joined #openstack-infra | 20:41 | |
| anteaya | clarkb: 652 through 658 had grokparsefailure | 20:41 |
| clarkb | anteaya: oh good, thats expected if we mismatch the log type and its tag | 20:41 |
| anteaya | wonderful | 20:42 |
| clarkb | pabelanger: doc, docs, documentation iirc | 20:42 |
| anteaya | yay I got as expected results | 20:42 |
| anteaya | clarkb: what now? | 20:42 |
| clarkb | pabelanger: one big benefit to the current setup is it means that projects can't get funny with how they builddocs | 20:43 |
| clarkb | pabelanger: everyone has to build with the standard command to do so | 20:43 |
| mmedvede | clarkb: thank you for the guide, having a working end-to-end debug env is useful. So the gist is every mode of logstash parsing needs to be tested? | 20:43 |
| clarkb | anteaya: talking about the doc tox targets | 20:43 |
| clarkb | mmedvede: yup | 20:43 |
| mordred | greghaynes: +billion | 20:43 |
| clarkb | mmedvede: anteaya I don't necessarily expect you to do all of that owkr but if you want to that would be great | 20:43 |
| clarkb | I think there are like 5 cases | 20:43 |
| anteaya | cool | 20:44 |
| clarkb | console, oslofmt, apache (swift), syslog, and maybe its only 4 | 20:44 |
| anteaya | well I would like to take a walk and then try the other cases | 20:44 |
| anteaya | and then compile my notes | 20:44 |
| anteaya | clarkb: where would you like my notes to live? | 20:44 |
| clarkb | anteaya: probably in the puppet-log_processor repo? | 20:45 |
| mmedvede | clarkb: I did not have much time budget to spend on it fulltime. I'll tinker with it for a bit | 20:45 |
| anteaya | clarkb: I'll aim for that | 20:45 |
| pabelanger | clarkb: Ya, just surprising. Fixed tox -edocs was standard now | 20:45 |
| *** tonytan4ever has quit IRC | 20:45 | |
| anteaya | mmedvede: I'll put them together and then ping you when I have something for review | 20:45 |
| anteaya | mmedvede: is that fair? | 20:45 |
| clarkb | pabelanger: fixed what? | 20:45 |
| pabelanger | s/Fixed/Figured | 20:45 |
| pabelanger | my bad | 20:45 |
| clarkb | ah | 20:46 |
| mmedvede | anteaya: wfm | 20:46 |
| clarkb | pabelanger: so ya part of the problem with that is people tried to do crazy funny things in their docs target | 20:46 |
| clarkb | pabelanger: meaning you couldn't just build_sphinx | 20:46 |
| anteaya | mmedvede: thanks, was nice to work with you on this | 20:46 |
| clarkb | which is a problem | 20:46 |
| * pabelanger nods | 20:46 | |
| clarkb | so I believe current stance is to not change how we build docs | 20:46 |
| mmedvede | anteaya: clarkb: I did compile condenced irc log, can post it if interested | 20:46 |
| clarkb | mmedvede: that would be great | 20:46 |
| anteaya | mmedvede: that would be nice too | 20:46 |
| mmedvede | clarkb: anteaya: post it to paste or to etherpad? | 20:47 |
| anteaya | mmedvede: hey would you like to post to the learn how to debug the gate mailing list thread? | 20:47 |
| anteaya | paste and then link to the mailing list is my choice | 20:47 |
| anteaya | clarkb: what do you think? | 20:47 |
| clarkb | that wfm | 20:47 |
| clarkb | I don't have much of a preference | 20:47 |
| mmedvede | anteaya: hmm, that would require quite a bit more editing | 20:47 |
| anteaya | mmedvede: is that doable for you? | 20:47 |
| *** thorst has quit IRC | 20:47 | |
| anteaya | mmedvede: oh well whatever you like | 20:47 |
| anteaya | the point of learning how to debug the gate is people work to learn | 20:48 |
| anteaya | it isn't handed to them | 20:48 |
| *** thorst has joined #openstack-infra | 20:48 | |
| anteaya | since they need the skill of digging to debug the gate | 20:48 |
| *** thorst has quit IRC | 20:49 | |
| mmedvede | anteaya: clarkb: ok, I'll polish it a bit more, and post it to paste/link from mailing list | 20:49 |
| *** thorst has joined #openstack-infra | 20:49 | |
| *** thorst has quit IRC | 20:49 | |
| mordred | jeblair: looks like ovh is seeing a climbing time for trying to delete servers | 20:50 |
| *** thorst has joined #openstack-infra | 20:50 | |
| anteaya | mmedvede: thanks so much | 20:50 |
| anteaya | mmedvede clarkb nice to have many different voices on the learning how to debug the gate thread | 20:51 |
| anteaya | clarkb: thanks very much for your time and your patience | 20:51 |
| anteaya | clarkb: I appreciate it | 20:51 |
| clarkb | no problem | 20:51 |
| mmedvede | anteaya: clarkb: thank you | 20:51 |
| anteaya | mmedvede: nice to work with you | 20:51 |
| anteaya | going for a walk now | 20:51 |
| *** shardy_ has joined #openstack-infra | 20:52 | |
| pleia2 | anteaya: enjoy | 20:52 |
| *** thorst_ has joined #openstack-infra | 20:53 | |
| *** rebase has joined #openstack-infra | 20:53 | |
| *** rpodolyaka1 has quit IRC | 20:53 | |
| *** thorst has quit IRC | 20:54 | |
| *** otter768 has joined #openstack-infra | 20:54 | |
| *** jogo has quit IRC | 20:55 | |
| *** Sukhdev has quit IRC | 20:55 | |
| *** jogo has joined #openstack-infra | 20:55 | |
| *** jogo has quit IRC | 20:55 | |
| *** jogo has joined #openstack-infra | 20:55 | |
| *** dimtruck is now known as zz_dimtruck | 20:55 | |
| tonyb | puppet folks, can you help me understand what's wrong with: https://review.openstack.org/#/c/246778/ , specifically the centos6 failure .... | 20:55 |
| clarkb | tonyb: its probably the broken haproxy module | 20:56 |
| openstackgerrit | Merged openstack-infra/system-config: drop reference to openstack-relmgr-office https://review.openstack.org/241353 | 20:56 |
| clarkb | mordred: nibalizer Hunner have we decied how we want to fix that yet? | 20:56 |
| tonyb | clarkb: okay that's what it looked like but it's my first puppet module patch .... | 20:56 |
| nibalizer | what are we talking about? | 20:57 |
| clarkb | nibalizer: haproxy broke hpcloud centos6 tests | 20:57 |
| nibalizer | clarkb: why aren't we just landing thttps://review.openstack.org/#/c/245836/ | 20:57 |
| *** thorst_ has quit IRC | 20:57 | |
| clarkb | nibalizer: I don't know I have stayed out of it | 20:57 |
| clarkb | I know there was a revert and ther ewas talk of also fixing it and rolling forward | 20:57 |
| nibalizer | so all the git servers are centos7 now right? | 20:57 |
| clarkb | nibalizer: not the one that runs haproxy | 20:57 |
| Hunner | Bah, more breakage? | 20:58 |
| clarkb | Hunner: well same breakage | 20:58 |
| Hunner | orly | 20:58 |
| clarkb | I just havne't seen anyone say "this is how we want ot fix" | 20:58 |
| nibalizer | clarkb: ya i think we just revert | 20:58 |
| clarkb | I have seen multiple "this is a possible fix" | 20:58 |
| sdague | mordred / jeblair - the devstack patch is approved, if anyone wants to approve - https://review.openstack.org/#/c/247004/ | 20:58 |
| nibalizer | and Hunner has a patch in to haproxy to fix it | 20:58 |
| nibalizer | we can deploy that later | 20:58 |
| nibalizer | also if this broke the haproxy node we would know | 20:59 |
| Hunner | nibalizer: Isn't that what clarkb is saying is still breaking? | 20:59 |
| clarkb | nibalizer: what does new module get us? | 20:59 |
| *** otter768 has quit IRC | 20:59 | |
| nibalizer | clarkb: see the commmit message on the thing being reverted | 20:59 |
| nibalizer | but one of them is the cleaner refresh thing you wanted | 20:59 |
| sdague | I am going to be super curious to see what our overall fail rates turn into after that lands | 20:59 |
| *** zz_dimtruck is now known as dimtruck | 20:59 | |
| clarkb | ooh we can relaod | 20:59 |
| clarkb | that makes me want to roll forward with Hunner's fix :) | 21:00 |
| *** shardy_ has quit IRC | 21:00 | |
| nibalizer | Hunner: no we didn't pin to the sha you made for us | 21:02 |
| nibalizer | clarkb: we can do that to | 21:02 |
| nibalizer | i gotta do other stuff right now but I will return | 21:02 |
| *** openstack has joined #openstack-infra | 21:05 | |
| *** placko_ has quit IRC | 21:06 | |
| *** EricGonczer_ has quit IRC | 21:06 | |
| *** rossella_s has quit IRC | 21:07 | |
| *** rossella_s has joined #openstack-infra | 21:08 | |
| *** sdake has joined #openstack-infra | 21:08 | |
| *** e0ne has joined #openstack-infra | 21:09 | |
| *** dprince has joined #openstack-infra | 21:09 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Add bashate test to devstack-gate https://review.openstack.org/237803 | 21:09 |
| pabelanger | clarkb: Hmm, that is confusing :) Either way, curious why we didn't catch the failure before the code merged | 21:10 |
| *** rfolco has quit IRC | 21:10 | |
| clarkb | pabelanger: it apparently only affects hpcloud test nodes | 21:10 |
| clarkb | I don't know the details on how they are different enough to be a problem | 21:10 |
| *** jistr has joined #openstack-infra | 21:10 | |
| *** thorst has joined #openstack-infra | 21:11 | |
| *** elemoine has quit IRC | 21:12 | |
| *** thorst has quit IRC | 21:15 | |
| *** rpodolyaka1 has joined #openstack-infra | 21:16 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/devstack-gate: Support period in project names for branch override flag https://review.openstack.org/247180 | 21:18 |
| *** signed8bit_ZZZzz is now known as signed8bit | 21:19 | |
| *** rpodolyaka1 has quit IRC | 21:19 | |
| *** changbl has joined #openstack-infra | 21:19 | |
| mordred | it's ipv6 | 21:20 |
| mordred | hpcloud nodes do not have ipv6 node | 21:20 |
| mordred | hpcloud nodes do not have ipv6 addresses | 21:20 |
| mordred | this means that the ruby code path that's doing things in haproxy gets tripped differently | 21:20 |
| mordred | and when you combine that with ruby 1.8 vs. 2.0 diferences in api | 21:20 |
| mordred | you hit a place where the code in 2.0 does the right thing but hte code in 1.8 blows up when you don't have an ipv6 address defined on your system | 21:21 |
| openstackgerrit | Merged openstack/requirements: Bump ceilometerclient to 2.0.0 https://review.openstack.org/246277 | 21:22 |
| mordred | solutions include: always have ipv6 because future, stop using ruby 1.8 because future, use Hunner's patch to puppet-haproxy which works around past and revert to old haproxy module until Hunner's patch is released | 21:22 |
| openstackgerrit | Tony Breeds proposed openstack-infra/project-config: Add #openstack-meeting-cp to eavesdrop https://review.openstack.org/246628 | 21:22 |
| openstackgerrit | Tony Breeds proposed openstack-infra/project-config: Add temporary pin on jaraco.itertools https://review.openstack.org/247182 | 21:22 |
| *** abregman|afk is now known as abregman | 21:22 | |
| *** thorst has joined #openstack-infra | 21:24 | |
| mtreinish | infra-root: is there a way we can check the hardware specs of the subunit2sql trove node? | 21:24 |
| *** abregman is now known as abregman|afk | 21:24 | |
| mtreinish | even, like what's exposed from nova | 21:24 |
| mtreinish | just trying to estimate the run time for the upcoming migration | 21:25 |
| *** thorst_ has joined #openstack-infra | 21:25 | |
| *** peristeri has quit IRC | 21:25 | |
| clarkb | mtreinish: we can't shell to it I don't think, so only what mysql can see | 21:25 |
| *** tonytan4ever has quit IRC | 21:25 | |
| mtreinish | clarkb: but it's still a nova instance right? cant we do a get to see the flavor and other stuff? | 21:25 |
| clarkb | mtreinish: its all hidden from the user I think | 21:26 |
| mtreinish | SlickNik: ^^^ do you have any suggestions? | 21:26 |
| pleia2 | yeah, afaik we don't have any view into that | 21:27 |
| *** tonytan4ever has joined #openstack-infra | 21:28 | |
| *** thorst has quit IRC | 21:28 | |
| SlickNik | mtreinish: A trove "show" on the instance should tell you the flavor. And then a trove "flavor show" or "flavor list" will tell you the hw profile of the instance. | 21:28 |
| SlickNik | hw profile of that flavor* | 21:29 |
| mtreinish | SlickNik: cool, thanks | 21:29 |
| mtreinish | clarkb, pleia2: ^^^ | 21:29 |
| clarkb | SlickNik: and thats all via trove client/api not nova? | 21:29 |
| *** hashar has quit IRC | 21:29 | |
| pleia2 | SlickNik: "on the instance"? | 21:30 |
| Hunner | clarkb: So yeah, the haproxy bug should be fixed with https://github.com/puppetlabs/puppetlabs-haproxy/pull/205 and pinned in https://review.openstack.org/#/c/246540/ (and eventually released in the next release) | 21:30 |
| mtreinish | I assumed it was through trove client | 21:30 |
| openstackgerrit | Merged openstack-infra/project-config: add release notes jobs for nova https://review.openstack.org/242027 | 21:30 |
| SlickNik | clarkb: Yes, that's all supported via the trove API. | 21:30 |
| SlickNik | pleia2: "trove show <instance-name or id>" | 21:30 |
| pleia2 | SlickNik: oh, gotcha | 21:30 |
| pleia2 | clarkb: shall I see about installing the trove client on the puppetmaster? | 21:31 |
| pleia2 | (writing the patch) | 21:31 |
| *** e0ne has quit IRC | 21:31 | |
| *** Rockyg has quit IRC | 21:31 | |
| clarkb | pleia2: I would start with just a virtualenv | 21:32 |
| clarkb | I have one in my homedir for neutronclient | 21:32 |
| pleia2 | I am eating thick tomato soup and I kind of want to add pasta too it | 21:32 |
| pleia2 | clarkb: ah, ok | 21:32 |
| pleia2 | mtreinish: investigating | 21:32 |
| nibalizer | clarkb: hi im back | 21:32 |
| *** abregman|afk has quit IRC | 21:32 | |
| nibalizer | whats our status on the haproxy thing | 21:32 |
| openstackgerrit | Merged openstack/requirements: Bump senlinclient to 0.1.5 https://review.openstack.org/245859 | 21:33 |
| mtreinish | pleia2: cool, thanks | 21:33 |
| clarkb | nibalizer: I think we just ned to decide which fix we want and approve it | 21:33 |
| clarkb | revert is fine with me if we ant to do that its has the +2's I just am not sure why no one is approving | 21:33 |
| *** maishsk_ has joined #openstack-infra | 21:34 | |
| nibalizer | i think there may be a sense that I was driving, and I'm not, so that may be the issue | 21:34 |
| nibalizer | clarkb: im good to try to fail forward if you and Hunner want to | 21:34 |
| nibalizer | do we have a patch that uses the sha1? | 21:34 |
| clarkb | I do not know (I don't know what the sha1 is either) | 21:34 |
| jeblair | yes | 21:35 |
| *** maishsk has quit IRC | 21:35 | |
| *** maishsk_ is now known as maishsk | 21:35 | |
| *** gongysh__ has joined #openstack-infra | 21:35 | |
| tonyb | nibalizer: https://review.openstack.org/#/c/246540/ ? | 21:35 |
| Hunner | ^ | 21:35 |
| jeblair | tonyb, nibalizer: yep | 21:35 |
| yonglihe | any one know how to get jenkins's vote result for a patch set? thanks | 21:36 |
| *** dangers_away is now known as dangers | 21:36 | |
| tonyb | yonglihe: Can't you toggle CI in the web UI and find the links that jenkins posted? | 21:36 |
| Hunner | yonglihe: Are you in the version2 view? There is a "toggle ci" button at the bottom | 21:36 |
| yonglihe | Hunner: , i mean via API, to get the vote result, 1 or -1. thanks. | 21:37 |
| yonglihe | there is the way to do that, i don't know how. | 21:38 |
| jeblair | yonglihe: https://review.openstack.org/Documentation/rest-api.html | 21:38 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Fix broken stable-compat-jobs for oslo https://review.openstack.org/247188 | 21:39 |
| pleia2 | mtreinish: trimmed out some possibly secretish stuff from `trove show`, but basically this is what we can see: http://paste.openstack.org/show/479347/ | 21:39 |
| nibalizer | i am good to approve 246540 if other infra-root are | 21:39 |
| clarkb | nibalizer: sounds good | 21:39 |
| nibalizer | confirmed its the right sha | 21:39 |
| mtreinish | pleia2: heh, that's no fun :) | 21:39 |
| pleia2 | mtreinish: that's `trove show` and details about the flavor | 21:39 |
| nibalizer | kk | 21:39 |
| pleia2 | mtreinish :D | 21:39 |
| mtreinish | SpamapS: ^^^ you might be interested too | 21:40 |
| dims_ | sdague : lifeless : hoping https://review.openstack.org/247188 will do the trick of fixing the oslo master libs testing with stable/liberty | 21:40 |
| nibalizer | approved | 21:40 |
| mtreinish | SpamapS: how does that compare to your machine? | 21:40 |
| yonglihe | jeblair: i do a quick check, no jenkins api there, am i right? | 21:40 |
| mtreinish | heh, ok that doesn't tell us much :) | 21:40 |
| mtreinish | I should have waited for paste to load | 21:40 |
| pleia2 | heh, yeah :\ | 21:40 |
| *** jistr has quit IRC | 21:41 | |
| yonglihe | i looking for the implement behind this tool:http://ci-watch.tintri.com/project?project=nova&time=7+days | 21:41 |
| mtreinish | pleia2: well it's good to know the 8GB Instance flavor type indeed has 8GB of ram | 21:41 |
| pleia2 | mtreinish: hehe | 21:41 |
| pleia2 | truth in advertising \o/ | 21:41 |
| mtreinish | pleia2: one thing is we'll definitely have to increase the volume size (if possible) | 21:42 |
| pleia2 | mtreinish: yeah, I did notice that it's getting a bit tight there | 21:42 |
| mtreinish | the migration (https://review.openstack.org/#/c/235009/ ) makes a temp table of everything | 21:42 |
| sdague | dims_: I think job is fine, we don't need the regex change | 21:42 |
| mtreinish | pleia2: it should get a lot better post migration, the indexes are kinda out of control right now | 21:42 |
| sdague | dims_: so the project config change, but the d-g change is not needed I don't think | 21:42 |
| mtreinish | (they are bigger than the data) | 21:42 |
| pleia2 | mtreinish: yeah, I think I remember the summit discussion | 21:43 |
| dims_ | sdague : ok will abandon the tr with '.' to underscore | 21:43 |
| * pleia2 checks size options | 21:43 | |
| mtreinish | pleia2: iirc in that same session fungi said we have up to 500gb available now | 21:43 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Fix broken stable-compat-jobs for oslo https://review.openstack.org/247188 | 21:44 |
| clarkb | I rmember fungi saying that too | 21:44 |
| dims_ | sdague done. stand alone review ^^ | 21:44 |
| pleia2 | I'm still only showing 320 as the biggest in the web UI | 21:44 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add a shade launch-node https://review.openstack.org/234975 | 21:44 |
| *** sabeen1 has joined #openstack-infra | 21:44 | |
| pleia2 | ah, but if I go to "Increase Volume" it will let me grow it to 500 | 21:45 |
| sdague | dims_: and then just make sure the 2 tr stanzas match | 21:45 |
| dims_ | sdague : ack. already done | 21:45 |
| sdague | yep, lgtm | 21:45 |
| *** sabeen3 has quit IRC | 21:45 | |
| *** amrith is now known as _amrith_ | 21:45 | |
| dims_ | sdague thanks for the help! tricky business :) | 21:46 |
| *** e0ne has joined #openstack-infra | 21:47 | |
| *** kozhukalov_ has joined #openstack-infra | 21:47 | |
| *** tiswanso has quit IRC | 21:50 | |
| *** MarkAtwood has joined #openstack-infra | 21:50 | |
| *** e0ne has quit IRC | 21:50 | |
| *** tiswanso has joined #openstack-infra | 21:50 | |
| *** tiswanso has quit IRC | 21:51 | |
| *** yamahata has joined #openstack-infra | 21:51 | |
| *** julim has quit IRC | 21:51 | |
| *** ayoung has quit IRC | 21:54 | |
| Swanson | Gots a lot of these .... | 21:54 |
| jhesketh | Morning | 21:54 |
| Swanson | ERROR zuul.GerritEventConnector: Received unrecognized event type 'topic-changed' from Gerrit. Can not get account information. | 21:54 |
| Swanson | That a problem or normal? | 21:55 |
| *** tsg has joined #openstack-infra | 21:56 | |
| *** salv-orl_ has joined #openstack-infra | 21:57 | |
| *** salv-orlando has quit IRC | 21:57 | |
| jeblair | Swanson: normal -- it means zuul needs to be updated to know about that event type. in the mean time, there will be some missing information in the unlikely event you use it as a trigger | 21:58 |
| jeblair | (you won't be able to trigger off of topic-changed events from specific users) | 21:58 |
| *** kgiusti has quit IRC | 21:58 | |
| *** austin81 has left #openstack-infra | 21:59 | |
| *** sridhar_ram has joined #openstack-infra | 22:00 | |
| Swanson | jeblair, cool. Thank you! | 22:00 |
| *** bswartz has quit IRC | 22:00 | |
| *** dkranz has joined #openstack-infra | 22:01 | |
| notmyname | looks like openstack meeting bot crashed/left/restarted(?) halfway through our meeting in #openstack-meeting. can someone with ops power take a look? | 22:01 |
| mattoliverau | Because it came back mid meeting, I think it's confused and assumes it's not in a meeting anymore. | 22:02 |
| *** changbl has quit IRC | 22:02 | |
| *** leong has joined #openstack-infra | 22:02 | |
| mattoliverau | We may need to start then end meeting.. I guess we've lost the swift meeting minutes tho | 22:03 |
| jeblair | notmyname, mattoliverau: if so you can startmeeting again. the minutes will be incomplete, though the channel log is still there | 22:03 |
| jeblair | i'll check the server log | 22:03 |
| dims_ | anteaya : if you get a chance please take a look at https://review.openstack.org/#/c/247188/ | 22:03 |
| sdague | jeblair: you want to +3 the d-g change now - https://review.openstack.org/#/c/247004/ - the devstack one is in the gate | 22:03 |
| *** hdd has joined #openstack-infra | 22:05 | |
| pleia2 | jeblair: looks like https://review.openstack.org/241353 was merged, which we should do during meetings | 22:05 |
| jeblair | sdague: done, thx | 22:05 |
| jeblair | INFO 2015-11-18T21:02:53 supybot Exiting due to Signal #15. | 22:05 |
| jeblair | pleia2: yep | 22:05 |
| jeblair | werr, 'not do' | 22:06 |
| pleia2 | but it was rechecked, so "oops" no one's fault really | 22:06 |
| pleia2 | er, right, not | 22:06 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-infracloud: Add bifrost manifest https://review.openstack.org/231757 | 22:06 |
| crinkle | greghaynes: ^ | 22:07 |
| greghaynes | oooo | 22:07 |
| greghaynes | crinkle: ok, I just finished a meeting | 22:07 |
| crinkle | the number of times we're running ansible from puppet is starting to get silly | 22:07 |
| greghaynes | crinkle: I have a bit to poke at things | 22:07 |
| crinkle | greghaynes: cool | 22:07 |
| greghaynes | crinkle: hah | 22:07 |
| greghaynes | crinkle: maybe we should find a way to just... run ansible :p | 22:07 |
| crinkle | greghaynes: ofc | 22:08 |
| greghaynes | crinkle: because it is actually running ansible from puppet from ansible | 22:08 |
| crinkle | greghaynes: yes | 22:08 |
| *** hdd has quit IRC | 22:08 | |
| crinkle | more config mgmt -> better devops | 22:08 |
| crinkle | i'm pretty sure | 22:09 |
| greghaynes | crinkle: wow | 22:09 |
| mordred | jeblair: how did your node launch run? | 22:09 |
| jeblair | it succeeded | 22:10 |
| mordred | yay! | 22:10 |
| mordred | jeblair: also, I could not find errors in nodepool logs to correspond to the red on the grpahs | 22:10 |
| *** sabeen1 has quit IRC | 22:10 | |
| jeblair | pypi.nyj01.openstack.org. 3600INA74.217.28.53 | 22:10 |
| mordred | woot! | 22:10 |
| anteaya | yay | 22:10 |
| *** doug-fish has quit IRC | 22:10 | |
| mordred | jeblair: I have a patch coming for shade to do the network and boot-from-volume stuff properly | 22:10 |
| jeblair | mordred: yeah, i just started looking into that, and all i see are some fip exhaustion | 22:10 |
| greghaynes | crinkle: so, is there some bit of work I could help with, looks like you got something set up to gen the dhcp config | 22:11 |
| openstackgerrit | Merged openstack-infra/system-config: Pin puppetlabs-haproxy to commit containing 1.8 fix https://review.openstack.org/246540 | 22:12 |
| crinkle | greghaynes: if bifrost was able to do that on its own that make it less weird | 22:12 |
| greghaynes | crinkle: ok, I'll poke | 22:12 |
| crinkle | greghaynes: alternatively if glean isn't that far away from being usable that would be better | 22:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: boot-from-volume and network params for server create https://review.openstack.org/247205 | 22:12 |
| mordred | jeblair: ^^ | 22:12 |
| mordred | jeblair: that's basically taking the stuff from launch-node and putting it in here sanely | 22:13 |
| greghaynes | crinkle: the issue is we also need to make the init script to set up networking how openstack needs it | 22:13 |
| anteaya | tonyb: looks good | 22:13 |
| greghaynes | crinkle: and I kind of think that would be easier to solve after we have a working system rather than having to work on init scripts from the start | 22:13 |
| anteaya | dims_: jenkins is unhappy: https://review.openstack.org/#/c/247188/ | 22:14 |
| crinkle | greghaynes: okay | 22:14 |
| crinkle | greghaynes: if you think 231757 is good enough then maybe we shouldn't spend more time poking bifrost and just start getting things merged | 22:15 |
| jeblair | mordred: oh, i found the rest of the errors, they rotated out into debug.log.2015-11-18_10 | 22:17 |
| greghaynes | crinkle: I like merging things | 22:17 |
| greghaynes | crinkle: let me review | 22:17 |
| dims_ | anteaya : does the rest of the structure look ok? y, fixing it now | 22:17 |
| greghaynes | crinkle: any thoughts on adding the baremetal node to site.pp? | 22:18 |
| jeblair | mordred: yeah, they are all fip exhaustion | 22:19 |
| *** claudiub has joined #openstack-infra | 22:19 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Fix broken stable-compat-jobs for oslo https://review.openstack.org/247188 | 22:20 |
| dims_ | anteaya : re-ran the "tox -e zuul" locally before pushing it this time :) | 22:20 |
| anteaya | yonglihe: http://git.openstack.org/cgit/openstack-infra/ciwatch/ | 22:20 |
| crinkle | greghaynes: there is https://review.openstack.org/#/c/208751/ but tests won't pass until we figure out what to do with the mysql module | 22:20 |
| *** vishpat has joined #openstack-infra | 22:20 | |
| vishpat | Hi, I am trying to save contact information, for "review.openstack.org" account and am hitting an error saying "Cannot save contact information" | 22:20 |
| yonglihe | anteaya: very thanks, that is what i'm looking for | 22:21 |
| clarkb | vishpat: see if https://ask.openstack.org/en/question/56720/cannot-store-contact-information-when-updating-info-in-openstack-gerrit/ gets it sorted out | 22:21 |
| greghaynes | crinkle: ok. I really dont know if theres anything I can do to help with that :/ | 22:21 |
| anteaya | yonglihe: welcome | 22:21 |
| anteaya | dims_: :) | 22:21 |
| zaro | mordred, jeblair anteaya : new info on gerrit rollback: https://review.openstack.org/#/c/245598 | 22:21 |
| greghaynes | crinkle: but that seems like the thing to hack on. Once we run this from site.pp and merge the manifests then were in a much more awesome spot | 22:21 |
| anteaya | zaro: woooo, thank you | 22:21 |
| crinkle | greghaynes: agreed | 22:22 |
| zaro | i completely forgot a major step in testing | 22:22 |
| *** tdurakov has joined #openstack-infra | 22:22 | |
| zaro | i'm looking into the reindex issue now. | 22:23 |
| anteaya | dims_: peace be with you | 22:23 |
| anteaya | zaro: glad you remembered it now | 22:23 |
| jeblair | mordred: though i don't understand how we can behitting fip exhaustion with 20 instances on a /27. | 22:23 |
| tdurakov | hi, everyone, got question about multinode env, which devstack-gate creates, is it possible to open port between primary and subsides with smth like iptables -A INPUT -p tcp --dport 111 -j ACCEPT | 22:24 |
| *** fitoduarte has quit IRC | 22:24 | |
| tdurakov | *subnodes | 22:24 |
| *** vishpat has quit IRC | 22:25 | |
| ianw | sudo dnf install libjpeg-turbo-devel <- this seems odd to get zuul unittests working | 22:25 |
| * ianw doesn't think about it too much | 22:25 | |
| mordred | jeblair: that's 'fascinating' | 22:25 |
| *** vpati011 has joined #openstack-infra | 22:26 | |
| *** Sukhdev has joined #openstack-infra | 22:26 | |
| *** dkranz has quit IRC | 22:26 | |
| nibalizer | clarkb: pin is in | 22:26 |
| clarkb | crinkle: nibalizer greghaynes weren't we going to update the apply tests to use the modules in two passes so that it would work? | 22:27 |
| greghaynes | crinkle: hrm, does your puppet manifest add the veth pair needed for openstack? | 22:27 |
| clarkb | then work on upgrading mysql module? (the jeepyb changes seem to be in at least so that is moving forward) | 22:27 |
| greghaynes | crinkle: or should we also bake the veth pair into https://review.openstack.org/#/c/231757/13/files/elements/infra-cloud-networking/static/etc/network/interfaces.d/eth2.cfg | 22:27 |
| *** MarkAtwood has quit IRC | 22:27 | |
| crinkle | greghaynes: https://review.openstack.org/#/c/231631/ adds the veth pairs and that apparently got merged, should have WIP'd it :( | 22:28 |
| crinkle | clarkb: i think nibalizer and i are unclear on what is involved in updating the apply tests | 22:29 |
| greghaynes | crinkle: I think thats fine if it happens as a result of puppet? | 22:29 |
| crinkle | greghaynes: the bridge is added as part of the infra-cloud-networking element, and adding the veth pairs doesn't disrupt the network afaict | 22:30 |
| crinkle | it is kind of strange though | 22:30 |
| greghaynes | crinkle: yea, so it should work | 22:30 |
| greghaynes | crinkle: yep | 22:30 |
| clarkb | crinkle: nibalizer the existing test run uses the little comment tags to select hosts. I owuld add a new comment thats says "infra-cloud" or something then after the existing trusty (or whatever distro) runs complete reintsall modules using the other module listing and run the infra cloud tests agains that distro | 22:31 |
| clarkb | and that change will be self testing | 22:31 |
| openstackgerrit | Sergey Skripnick proposed openstack/requirements: Bump ddt version to >=1.0.1 https://review.openstack.org/247214 | 22:31 |
| clarkb | I think most of that is done since nibalizre wrote the stuff to test it | 22:32 |
| clarkb | it just needs to run in one job and be self testing | 22:32 |
| crinkle | clarkb: okay i think that makes sense | 22:32 |
| *** jpeeler has quit IRC | 22:32 | |
| crinkle | clarkb: what about for actually deploying it? right now we have one puppetmaster in one environment so we can't have both versions installed | 22:32 |
| anteaya | dims_: today is not your day apparently | 22:33 |
| *** sabeen1 has joined #openstack-infra | 22:33 | |
| mordred | jeblair: do we have hidden floating ips somewhere? | 22:33 |
| clarkb | crinkle: I thought the plan for that was to use puppet apply in infra cloud but maybe the timeline for that isn't conducive to progress on clouding? | 22:33 |
| clarkb | crinkle: but the idea there was avoid the master entirely I think (due to that reason) | 22:34 |
| *** maishsk_ has joined #openstack-infra | 22:34 | |
| crinkle | clarkb: i thought you had objections to that because it would make it different from the rest of infra | 22:34 |
| clarkb | crinkle: as an alternative we could setup a different puppet environment for infra cloud during the time they are different | 22:34 |
| clarkb | crinkle: yes, the goal should be to make them the same | 22:34 |
| clarkb | crinkle: and the plan is to stop puppetmastering across hte board | 22:34 |
| openstackgerrit | Merged openstack/requirements: Raise upper-constraints on python-novaclient to 2.35.0 https://review.openstack.org/244778 | 22:34 |
| openstackgerrit | Merged openstack/requirements: Raise upper-constraints on python-novaclient to 2.35.0 https://review.openstack.org/244778 | 22:34 |
| jeblair | mordred: i just checked all our tenants (so much clicking! why do i have to do that?!) and i didn't see any laying around. openstackjenkins had 3 unassigned, so i released them. but that should be within our buffer (we should still have had 4 extra). | 22:34 |
| *** rick_ has joined #openstack-infra | 22:35 | |
| *** gongysh__ has quit IRC | 22:35 | |
| anteaya | dims_: you have managed to undefine a bunch of existing jobs: http://logs.openstack.org/88/247188/3/check/gate-project-config-layout/92941d4/console.html#_2015-11-18_22_26_37_721 | 22:35 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul: Use 127.0.0.1 rather than localhost for statsd host https://review.openstack.org/229247 | 22:35 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul: Update to statsd 3.0 https://review.openstack.org/229232 | 22:35 |
| nibalizer | puppet apply is pretty darn close | 22:35 |
| greghaynes | clarkb: crinkle If I parse correctly - I would *really* like not push back having the infra-cloud puppet runs kicked off by infra so we can move out of being a hidden working group | 22:35 |
| *** sabeen3 has joined #openstack-infra | 22:35 | |
| *** hemna has quit IRC | 22:35 | |
| *** jpeeler has joined #openstack-infra | 22:36 | |
| clarkb | greghaynes: ya thats not the plan | 22:36 |
| greghaynes | er, I would like to not push back | 22:36 |
| clarkb | greghaynes: basically puppet apply run by ansible instead of puppet agent | 22:36 |
| nibalizer | we could also use a puppet environment maybe | 22:36 |
| greghaynes | ah, ok | 22:36 |
| clarkb | greghaynes: but still triggered from ansible | 22:36 |
| *** leong has quit IRC | 22:36 | |
| *** alivigni has quit IRC | 22:36 | |
| jeblair | what's still needed for global masterless puppet? the hiera split was the hard part, right? :) | 22:37 |
| crinkle | clarkb: nibalizer okay since puppet-apply is pretty close to being a real thing i think it would be okay for infra-cloud to start off using it | 22:37 |
| *** maishsk has quit IRC | 22:37 | |
| jeblair | (maybe dusting off the old puppet apply patches?) | 22:37 |
| *** maishsk_ is now known as maishsk | 22:37 | |
| nibalizer | i have a test enviorment with puppet-apply being driven from run_all.sh | 22:37 |
| *** lykinsbd has quit IRC | 22:37 | |
| *** sabeen1 has quit IRC | 22:37 | |
| tonyb | anteaya: Thanks | 22:38 |
| *** tsg has quit IRC | 22:38 | |
| anteaya | tonyb: it looks good! nice work there! | 22:38 |
| anteaya | tonyb: tests and everything | 22:38 |
| nibalizer | I think we want to manage puppet.conf and hiera.yaml from ansible | 22:39 |
| tonyb | anteaya: I have to admit that I'm not 100% certain it'll work but apart from the tests I don't know what else to do | 22:39 |
| nibalizer | instead of from-puppet | 22:39 |
| nibalizer | and I think agreeing to that will help move puppet-apply over the edge | 22:39 |
| tonyb | anteaya: to be clear I know the code works but I can't test the dynamic reload of the plugin/config | 22:39 |
| nibalizer | if you think about it we need puppet to run using config files, and it can't both be responsible for those files and use them | 22:40 |
| nibalizer | so having ansible lay down the correct configuration files and kick of a puppet run makes a lot of sense | 22:40 |
| *** tonytan4ever has quit IRC | 22:40 | |
| clarkb | nibalizer: is that something we sould add to the ansible puppet module? | 22:40 |
| nibalizer | clark proposed that we just rely on puppet defaults, and I like that more, but I think our configuration needs are too much to use the defaults | 22:40 |
| clarkb | right | 22:40 |
| nibalizer | clarkb: i started that work here | 22:40 |
| nibalizer | https://review.openstack.org/#/c/243399/ | 22:40 |
| anteaya | tonyb: yeah that makes sense | 22:41 |
| jeblair | and we can't simplify our configuration? :) | 22:41 |
| * tonyb probably should have uesed the same topic for all of the changes to make it easier for review | 22:41 | |
| nibalizer | thats very much a rough draft | 22:41 |
| clarkb | nibalizer: because of the hiera stuff iirc? | 22:41 |
| *** daniloramalho has quit IRC | 22:41 | |
| anteaya | tonyb: you can change the topics to match | 22:41 |
| nibalizer | clarkb: mostly | 22:41 |
| jeblair | nibalizer: at first consideration, ansible-puppet being responsible for puppet config files makes sense to me. | 22:41 |
| jeblair | mordred: ^ | 22:41 |
| nibalizer | if we wanted ansible to manage something like graphite, it would set up the graphite config files | 22:42 |
| clarkb | jeblair: ya (I still wish we could default but if we can't this is the next best thing IMO) | 22:42 |
| *** daniloramalho has joined #openstack-infra | 22:42 | |
| jeblair | clarkb: agreed | 22:42 |
| tonyb | anteaya: Yeah, actually I have 2 topics which is right as the changes aren't really related you couls land all of either topic without needing/affecting the other | 22:42 |
| jeblair | nibalizer: this will probably be less mind-bendy in the long run too :) | 22:42 |
| * tonyb => coffee machine | 22:43 | |
| anteaya | tonyb: okay, as long as you are happy, and you know that you can edit topics and commit messages in the gui | 22:43 |
| anteaya | tonyb: enjoy | 22:43 |
| nibalizer | ya | 22:44 |
| nibalizer | im going to get dinner, but i like where this conversation is headed | 22:44 |
| mordred | jeblair, nibalizer: just got back | 22:44 |
| greghaynes | crinkle: one more question - with the puppet element you added, any reason to not just use the puppet element in system-config? | 22:44 |
| nibalizer | is there something in ansible thhat lets us feed our specific configuration of config files to the more general anislbe-puppet role? | 22:44 |
| mordred | SO ... I'm sorry for not being here for the conversation so far ... | 22:45 |
| mordred | but I do not think we need this to move forward | 22:45 |
| mordred | although I think it's cool | 22:45 |
| greghaynes | crinkle: er, project-config | 22:46 |
| *** amitgandhinz has quit IRC | 22:46 | |
| mordred | I think we can have a role which copies hiera files from our hard-coded hierarchy to get us off of a puppetmaster | 22:46 |
| mordred | and that gives us more room to figure out how to do slick parameterization in ansible | 22:46 |
| *** jaypipes has joined #openstack-infra | 22:47 | |
| mordred | if nobody else can use this role other than us when we go live, I tink that' ok | 22:47 |
| jeblair | mordred: do you have a patch for that? | 22:47 |
| clarkb | mordred: its not the hiera files | 22:47 |
| clarkb | mordred: its the puppet config | 22:47 |
| mordred | https://review.openstack.org/#/c/165157/ | 22:47 |
| adreznec_ | Hey all, quick question: could somebody take a look this project-config review for me when they have a minute? https://review.openstack.org/#/c/245232/ It has 3 +2s and a Workflow+1 but I can't get Jenkins gate to run against it. I tried a recheck, but no luck. | 22:47 |
| mordred | clarkb: we do not need to solve the puppet config | 22:47 |
| clarkb | mordred: according to nibalizer we do | 22:47 |
| mordred | clarkb: it is not a problem right now | 22:47 |
| clarkb | mordred: because of hiera | 22:47 |
| mordred | nope | 22:48 |
| mordred | hiera is already configured | 22:48 |
| mordred | those files are already there | 22:48 |
| clarkb | mordred: since you have to configure it to know where to look and the hierarchy | 22:48 |
| mordred | nonono | 22:48 |
| clarkb | mordred: those files are applied by puppet | 22:48 |
| mordred | we have the outcome of our config hardcoded into the current ansible | 22:48 |
| clarkb | mordred: so there is a chicken and egg | 22:48 |
| mordred | we do NOT need a thing which can read a general hiera config | 22:48 |
| *** burgerk_ has joined #openstack-infra | 22:48 | |
| clarkb | oh | 22:48 |
| mordred | and figure out a general hierarchy | 22:48 |
| clarkb | ok I misparsed what you said | 22:48 |
| clarkb | I thought you were saying copy the hiera split out files | 22:48 |
| clarkb | but you are saying copy the hiera config for hiera hierarchy | 22:48 |
| mordred | no | 22:49 |
| mordred | I am saying what you thought | 22:49 |
| clarkb | mordred: ok that wont work | 22:49 |
| mordred | I am saying we copy the split out hiera files | 22:49 |
| mordred | why not? | 22:49 |
| clarkb | mordred: because hiera won;t lookup our data properly | 22:49 |
| mordred | why not? | 22:49 |
| clarkb | mordred: because we are using a non default hiera hierarchy | 22:49 |
| mordred | we have the hiera config files for that already applied | 22:49 |
| clarkb | which needs to be configured | 22:49 |
| *** burgerk has quit IRC | 22:49 | |
| mordred | it's live in production right now | 22:49 |
| clarkb | mordred: where? | 22:49 |
| crinkle | greghaynes: the puppet element in project-config depends on a bunch of other elements that don't make sense for a non-nodepool node | 22:49 |
| mordred | it's been live for like a month | 22:49 |
| *** edmondsw has quit IRC | 22:49 | |
| clarkb | (in puppet) | 22:49 |
| clarkb | this is the problem | 22:49 |
| mordred | one sec ... I have to find it | 22:49 |
| greghaynes | crinkle: awesome | 22:49 |
| mordred | it's in puppet, it's live | 22:50 |
| clarkb | mordred: yup | 22:50 |
| clarkb | mordred: thats the issue | 22:50 |
| clarkb | mordred: so you spin up a new host | 22:50 |
| clarkb | mordred: you have to configure this before running puppet | 22:50 |
| *** ayoung has joined #openstack-infra | 22:50 | |
| crinkle | greghaynes: i could be mistaken but i think production nodes install puppet via the launch_node.py script, not that element | 22:50 |
| clarkb | the suggestion is to have ansible do it | 22:50 |
| clarkb | because otherwise puppet can't run | 22:50 |
| *** gordc has quit IRC | 22:50 | |
| jeblair | crinkle: correct | 22:50 |
| clarkb | mordred: I had the same argument with nibalizer | 22:51 |
| clarkb | I wanted it to just work but I am pretty sure he is correct and we need something a bit richer in ansible | 22:51 |
| jeblair | crinkle, greghaynes: launch-node copies system-config/install_puppet.sh to the new machine and runs that | 22:51 |
| greghaynes | crinkle: Yea, this is definitely a new thing were doing | 22:51 |
| greghaynes | production node from a dib image | 22:51 |
| mordred | http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/puppet/hiera.yaml | 22:51 |
| mordred | it's right there | 22:51 |
| mordred | clarkb: ^^ | 22:51 |
| clarkb | mordred: yes thats the problem | 22:52 |
| jeblair | greghaynes: you may be interested in https://review.openstack.org/140840 | 22:52 |
| mordred | what is the problem? it's there. it's applied? | 22:52 |
| clarkb | mordred: it is in puppet but puppet depends on it | 22:52 |
| crinkle | jeblair: do you think it would make sense to have https://review.openstack.org/#/c/231757/13/files/elements/puppet/install.d/05-puppet exist somewhere in system-config? | 22:52 |
| clarkb | mordred: you have to lay that down before you run puppet | 22:52 |
| clarkb | mordred: right now its fine bceause its on the puppetmaster | 22:52 |
| greghaynes | jeblair: oooo | 22:52 |
| clarkb | mordred: so its done for every host | 22:52 |
| clarkb | mordred: but as soon as you move to puppet apply a new host wont have that config and puppet wont work | 22:52 |
| crinkle | jeblair: oh now i see the project-config patch :) | 22:53 |
| clarkb | for existing hosts it may work if you sequence the steps properly | 22:53 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:53 | |
| mordred | clarkb: k. nod. it's a static file though -so we can hack around that for a second by having launch-node wget it | 22:53 |
| clarkb | but can break if not done in proper sequence | 22:53 |
| mordred | it's just a single static file | 22:53 |
| mordred | we don't need ansible templating of creation of the file | 22:53 |
| clarkb | mordred: right so suggestion is have ansible write it out | 22:53 |
| mordred | I think I'm mostly saying why don't we do simple first | 22:53 |
| zaro | clarkb: so can you explain to me what the rewrite rule 'RewriteRule ^/(.*)$ http://localhost:8081/$1 [NE,P]' does? or what we use it for? | 22:53 |
| mordred | before we try to make it dynamic and configurable | 22:53 |
| *** burgerk_ has quit IRC | 22:54 | |
| *** mriedem has quit IRC | 22:54 | |
| clarkb | mordred: sure we could make a middle step of do the hack | 22:54 |
| zaro | clarkb: why it's important | 22:54 |
| *** dingyichen has joined #openstack-infra | 22:54 | |
| mordred | if we need to have ansible copy it in - fine - let's just put that one static file in the ansible and have it copy it | 22:54 |
| clarkb | zaro: it is importantbecause it makes all of the stuff above it work | 22:54 |
| jeblair | clarkb, mordred: or we can add that to install_puppet.sh if it's only needed for bootstrapping. | 22:54 |
| mordred | jeblair: yah | 22:54 |
| clarkb | zaro: in particular /p/ for git repo serving does not happen against the java daemon | 22:54 |
| clarkb | jeblair: mordred it is only needed for boostrapping if we continue to manage it in puppet too | 22:55 |
| clarkb | zaro: so that says if all o fthose conditions above are true then proxy the request to localhost port 8081 | 22:55 |
| *** mestery has quit IRC | 22:55 | |
| mordred | install_puppet comes from system-config, which is also where hiera.yaml is from | 22:55 |
| *** cbader has quit IRC | 22:55 | |
| clarkb | zaro: so it is deciding when to proxy to gerrit and hwne not to proxy to gerrit | 22:55 |
| *** otter768 has joined #openstack-infra | 22:55 | |
| *** unicell has quit IRC | 22:55 | |
| mordred | so whatever gets install_puppet can also get hiera.yaml from the puppet repo, and then then next time puppet is run if there has been a change, then puppet will update it to the new value, no? | 22:56 |
| mordred | then - as a follow up, once puppet apply is there and works nicely, we cna TOTALLY add the cool thing nibalizer wrote | 22:56 |
| mordred | because I like it | 22:56 |
| clarkb | mordred: yes that should work | 22:56 |
| mordred | it's just a lot of moving parts to be something that blocks an already long-lived and touchy migration because of the scope of the change | 22:56 |
| mordred | cool | 22:56 |
| clarkb | zaro: the ^/(.*)$ matches the url path and stores it in $1 then appends $1 to the proxied server path | 22:57 |
| openstackgerrit | Merged openstack/requirements: Rename networking-bagpipe-l2 to networking-bagpipe https://review.openstack.org/245625 | 22:57 |
| clarkb | zaro: the NE is the same as nocanon in your change and the P sas proxy | 22:57 |
| jeblair | openstack --os-cloud openstackci-internap volume type list | 22:58 |
| jeblair | | df249351-41b7-407d-9681-ec31eadb5d2e | solidfire0 | | 22:58 |
| mordred | jeblair: how fascinating | 22:58 |
| mordred | and yay! osc + occ is working for you :) | 22:58 |
| jeblair | mordred: yep! | 22:58 |
| mordred | jeblair: fwiw, you can also set OS_CLOUD= if you don't want to type that all the time | 22:59 |
| jeblair | mordred: well, i actually just dropped into interactive mode | 22:59 |
| jeblair | which is kinda handy | 22:59 |
| mordred | nice | 22:59 |
| mordred | jeblair, clarkb: https://review.openstack.org/#/c/241495/ is also needed - but I think it can also be done differently | 23:00 |
| zaro | clarkb: what or who uses /p/ ? | 23:00 |
| *** dprince has quit IRC | 23:00 | |
| mordred | I just havne't followed up on it | 23:00 |
| clarkb | zaro: all the things | 23:00 |
| clarkb | zaro: gerrit puts it in clone lines iirc | 23:00 |
| *** otter768 has quit IRC | 23:01 | |
| zaro | clarkb: so if i clone from git.o.o then i'm using /p/ ? | 23:02 |
| clarkb | zaro: no | 23:02 |
| clarkb | zaro: if you clone from https://review.openstack.org/p/openstack-infra/project-config for example | 23:03 |
| *** vpati011 has quit IRC | 23:03 | |
| *** Sukhdev has quit IRC | 23:04 | |
| *** amuller has quit IRC | 23:04 | |
| nibalizer | how about an ansible task that just wgets puppet.conf and hiera.yaml in ansible_puppet.yml before running puppet | 23:04 |
| zaro | clarkb: so i never do that. who does that? zuul? our clone for the jenkins slaves? | 23:04 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Add puppet apply settings to puppet.conf https://review.openstack.org/241495 | 23:04 |
| clarkb | zaro: anyone can | 23:04 |
| mordred | clarkb, jeblair, nibalizer: ^^ ok. that's what was I thinking there | 23:05 |
| nibalizer | right beforee it installs or updates all the modules | 23:05 |
| mordred | nibalizer: (sorry, I totally hijacked your change there) | 23:05 |
| *** Piet has quit IRC | 23:05 | |
| zaro | clarkb: it's not exposed or documented. so who does? | 23:05 |
| tdurakov | sdague, are you around? | 23:05 |
| clarkb | zaro: its a gerrit thing so its both documented and exposed I think | 23:05 |
| *** tsg has joined #openstack-infra | 23:05 | |
| clarkb | zaro: gerrit suports /p/ | 23:05 |
| clarkb | zaro: we don't let gerrit serve that path because it eats up threads | 23:06 |
| clarkb | instead we hvae apache serve it | 23:06 |
| mordred | nibalizer: we could do that - but I just don't think we need to to take the next step - install_puppet.sh is already a thing we run one at startup, and after that puppet alreayd manages the file | 23:06 |
| *** dangers is now known as dangers_away | 23:06 | |
| clarkb | zaro: lets be clear /p/ is not the issue here | 23:06 |
| clarkb | zaro: the issue is openid is broken in newer gerrit | 23:06 |
| clarkb | zaro: we should fix that | 23:06 |
| mordred | nibalizer: if we copy it every time we run ansible, we're introducting ansible config file management, which means now we have two config management systems | 23:06 |
| clarkb | I personally cannot fix it without getting a cla signed | 23:06 |
| mordred | nibalizer: which might be cool - but makes things more complex than we strictly need at the moment | 23:07 |
| clarkb | if it would help to do that I suppose I can figured out what I need to do | 23:07 |
| zaro | clarkb: thanks for the explanation. yeah, i agree with you. | 23:07 |
| jeblair | zaro, clarkb: there were 120488 requests for /p/ yesterday. | 23:07 |
| clarkb | jeblair: thanks | 23:07 |
| mordred | nibalizer: (I like your templated thing which would potentially allow us to grok the actual hierarchy in the role)) | 23:07 |
| zaro | i guess i need to read the gerrit docs again. | 23:07 |
| nibalizer | mordred: im gonna eat dinner bow | 23:08 |
| nibalizer | pick this up later? | 23:08 |
| tdurakov | clarkb, hi, got 2 minutes? question about d-g, with USE_SCREEN=False in localrc what will be simplest way to restart nova-compute service? | 23:10 |
| clarkb | tdurakov: devstack writes out the process ids I think so you would kill that process and start a new one then update the file of pids | 23:11 |
| tdurakov | clarkb, ok, thank you | 23:12 |
| tdurakov | clarkb, could you also give some feedback for: https://review.openstack.org/#/c/247081/ trying to set up nfs as a shared storage | 23:13 |
| mordred | nibalizer: yah | 23:14 |
| jeblair | mordred: initial bandersnatch run on nyj01 started | 23:14 |
| *** camunoz has joined #openstack-infra | 23:14 | |
| *** Sukhdev has joined #openstack-infra | 23:15 | |
| mordred | jeblair: woot! | 23:15 |
| *** rick_ has quit IRC | 23:16 | |
| anteaya | fridge! | 23:16 |
| jeblair | spoon! | 23:17 |
| pleia2 | anteaya: good luck with the stashing | 23:17 |
| *** sdake_ has joined #openstack-infra | 23:17 | |
| *** sdake has quit IRC | 23:18 | |
| clarkb | tdurakov you can use all when running the same stuff on primary and subnodes. Also for iptables I would consider only allowing connections between the hosts | 23:18 |
| clarkb | tdurakov: and finally don't forget to squashroot (which may be a default ) | 23:18 |
| *** cloudtrainme has quit IRC | 23:19 | |
| *** pradk has quit IRC | 23:19 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 23:20 | |
| tdurakov | clarkb, acked, thanks again! | 23:20 |
| *** bpokorny_ has joined #openstack-infra | 23:21 | |
| *** kozhukalov_ has quit IRC | 23:22 | |
| tdurakov | clarkb, am i right that primary node here is a current node/localhost? | 23:24 |
| openstackgerrit | Merged openstack-infra/devstack-gate: enable ebtables workaround in devstack https://review.openstack.org/247004 | 23:24 |
| clarkb | tdurakov: it is the node where jenkins starts the test execution yes | 23:24 |
| anteaya | pleia2: thank you | 23:24 |
| tdurakov | why it was decided to use ansible for that? | 23:25 |
| *** dingyichen has quit IRC | 23:25 | |
| tdurakov | clarkb, ^ | 23:25 |
| *** bpokorny has quit IRC | 23:25 | |
| anteaya | mmedvede: thank you for posting that! | 23:25 |
| *** dingyichen has joined #openstack-infra | 23:25 | |
| anteaya | mmedvede: nicely done | 23:26 |
| clarkb | tdurakov: because its really easy to classify your machines and just have it work over ssh with ansible | 23:26 |
| clarkb | tdurakov: and ssh is the only initial communication allowed | 23:26 |
| tdurakov | clarkb, ok | 23:27 |
| clarkb | tdurakov: so rather than write an ssh for loop tool just use ansible (it also installs via pip from our mirrors into a virtualenv so does not affect the test environment itself) | 23:27 |
| anteaya | mmedvede: I had thought you were just going to post a link to the irc logs, that looks like a great patch for the puppet-log_processer docs | 23:27 |
| *** ddieterly has quit IRC | 23:27 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Fix broken stable-compat-jobs for oslo https://review.openstack.org/247188 | 23:28 |
| *** xyang1 has quit IRC | 23:28 | |
| *** gongysh__ has joined #openstack-infra | 23:29 | |
| anteaya | mmedvede: that is really nicely done | 23:29 |
| mmedvede | anteaya: thanks! Would need to ask clarkb ' for permission to his words :) | 23:30 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Correct Diversity WG calendar entry The Diversity Working Group holds meetings weekly, at alternating times. The entry for the calendar was never accurate. This patch fixes the wrong times. It also now includes accurate IRC channel. https://review.openstack.org/244999 | 23:30 |
| clarkb | dims_: I am not sure I like that change | 23:30 |
| *** aysyd has quit IRC | 23:30 | |
| clarkb | dims_: zuul should always checkout the change under test | 23:30 |
| clarkb | dims_: so not sure what wasn't working | 23:31 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul: Update to statsd 3.0 https://review.openstack.org/229232 | 23:31 |
| clarkb | zuul/devstack-gate | 23:31 |
| anteaya | mmedvede: of course | 23:31 |
| clarkb | mmedvede: oh sure I don't claim any special ownership | 23:31 |
| anteaya | clarkb: well you do have copyright over everything you say | 23:32 |
| anteaya | unless you say it on facebook apparently | 23:32 |
| mmedvede | hehe | 23:32 |
| * anteaya doesn't say things on facebook | 23:32 | |
| dims_ | clarkb : please take a look at https://review.openstack.org/#/c/246404/ gate-tempest-dsvm-neutron-src-oslo.config-liberty zuul pulls from master just fine. but then ends up with stable/liberty branch (see pip freeze output for example) | 23:33 |
| *** smarcet1981 has quit IRC | 23:33 | |
| clarkb | dims_: if that is happening the other job is wrong | 23:33 |
| clarkb | dims_: and we should fix it rather than pasting over it with another job | 23:33 |
| dims_ | clarkb : apparently this scenario is not supported per sdague | 23:34 |
| dims_ | we had a long discussion over on #openstack-oslo | 23:34 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Added OSOps to the Monitoring and Tools WG https://review.openstack.org/247075 | 23:34 |
| clarkb | I think that is wrong :) | 23:34 |
| *** signed8bit_ZZZzz is now known as signed8bit | 23:34 | |
| clarkb | dims_: http://logs.openstack.org/04/246404/5/check/gate-tempest-dsvm-neutron-src-oslo.config-liberty/09ac4b1/logs/devstack-gate-setup-workspace-new.txt.gz#_2015-11-18_14_56_26_406 is the problem | 23:34 |
| *** ZZelle_ has quit IRC | 23:35 | |
| dims_ | clarkb we need stable/liberty everything running with oslo.config built from master | 23:35 |
| clarkb | dims_: so I thought that the checkout should always fallback on master | 23:35 |
| clarkb | dims_: yes I know I am saying thats how it should work, the fact it doens't work that way is a bug | 23:35 |
| clarkb | jeblair: ^ or am I mistaken? | 23:35 |
| dims_ | clarkb : since oslo.config has stable/liberty branch the harness picks that up over master | 23:36 |
| clarkb | dims_: its not supposed to | 23:36 |
| dims_ | clarkb what can i say...logs are the evidence :) | 23:36 |
| jeblair | clarkb, dims_: i've been activated. it will take me some time to understand this. :) | 23:36 |
| clarkb | dims_: my understanding of the process is try stable/liberty zuul ref if does not exist try master zuul ref (this does exist and should be checked out) | 23:36 |
| clarkb | dims_: yes I am not refuting thats how it works | 23:36 |
| clarkb | dims_: what I hvae concerns with is that instead of fixing the bug we are making a different job | 23:37 |
| clarkb | jeblair: basically if override branch is set to stable/kilo d-g will attempt stable/kilo zuul ref and fail because its a master change, d-g doesn't then try master zuul ref which I thought it should | 23:37 |
| *** fitoduarte has joined #openstack-infra | 23:37 | |
| dims_ | clarkb : i want to see if it will work before changing the job that already exists so we don't break everyone | 23:37 |
| clarkb | then if that fails fall back to the branch itself and if that fails fallback on master | 23:37 |
| *** tdurakov has quit IRC | 23:38 | |
| clarkb | dims_: we can test that with d-g and depends on | 23:38 |
| dims_ | clarkb i wanted to try setting OVERRIDE_OSLOCONFIG_PROJECT_BRANCH to master | 23:38 |
| clarkb | dims_: no need to make an entirely new job for every oslo project | 23:38 |
| dims_ | clarkb fair point, if you can give me a clue on what i can use to switch on that flag in the existing job-template i can definitely try that | 23:39 |
| jeblair | clarkb: i don't think it should try the master branch if override is set to stable/kilo -- i think that's the thing that is being overridden. | 23:40 |
| *** sdake_ has quit IRC | 23:40 | |
| *** sdake has joined #openstack-infra | 23:40 | |
| clarkb | dims_: you just push a change that hard sets the flag in d-g then depend on that in oslo.config change | 23:40 |
| clarkb | jeblair: I thought it would always fallback to master | 23:40 |
| clarkb | at least I believe that is zuul-cloners behavior | 23:41 |
| jeblair | clarkb: it should match | 23:41 |
| jeblair | it's often easier to think about less in terms of "a then b then c" and more in terms of specifying what branch should be checked out -- generally we check out the branch for the change under test (master in this case) unless you specify otherwise | 23:42 |
| jeblair | the branch-specific jobs specify otherwise | 23:42 |
| zaro | clarkb: do you happen to know where /p/ is documented? I cannot seem to find it anywhere in the gerrit docs. the closes i references to proxing is here: https://gerrit-review.googlesource.com/Documentation/config-reverseproxy.html | 23:43 |
| clarkb | jeblair: right and tahts fine in the case of no zuul ref but I thought always get available zuul ref was desireable? | 23:43 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Removed No longer attended OpsMeeting https://review.openstack.org/247070 | 23:43 |
| jeblair | clarkb: no, it's "get the proposed future state of the branch the user wants", which may or may not have any zuul refs | 23:43 |
| jeblair | dims_: so what do you want this job to do? | 23:44 |
| zaro | clarkb: i've looked in 2.8 docs as well but didn't see anything on it either. | 23:44 |
| clarkb | zaro: I guess its not documented by upstream then? | 23:47 |
| clarkb | zaro: its a valid path you can test it on your local gerrit | 23:47 |
| *** bpokorny_ has quit IRC | 23:47 | |
| zaro | clarkb: yeah, i know. i did test it. but i guess i had know idea that it existed. | 23:48 |
| *** bpokorny has joined #openstack-infra | 23:48 | |
| *** adduarte has joined #openstack-infra | 23:48 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Added weekly meeting for Neutron Upgrades subteam https://review.openstack.org/246949 | 23:48 |
| clarkb | jeblair: ok I see in zuul-cloner that in this case fallback branch and override branch would be the same so no master ref would be attempted | 23:48 |
| *** dimtruck is now known as zz_dimtruck | 23:48 | |
| zaro | just wanted to learn more about it, hoping that the docs would explain. | 23:48 |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr: Fail if no tests were successfully executed https://review.openstack.org/246980 | 23:49 |
| clarkb | dims_: so in that case I thinkyou need to be changing the existing job and not making a new one | 23:49 |
| clarkb | dims_: either way we don't want or need a new job I do't think | 23:50 |
| *** signed8bit is now known as signed8bit_ZZZzz | 23:50 | |
| clarkb | jeblair: oh wait no | 23:50 |
| clarkb | jeblair: it specifically has a check for zuul branch in zuul cloner | 23:50 |
| clarkb | jeblair: so I think my interpretation of this is what z-c does do | 23:50 |
| jeblair | clarkb: the unit tests are designed to ensure the behavior matches :/ | 23:51 |
| clarkb | oh wait it then replaces zuul branch with fallback branch | 23:51 |
| clarkb | that code is very confusing | 23:51 |
| jeblair | clarkb: i "simplified" it. :) | 23:51 |
| *** fitoduarte has quit IRC | 23:51 | |
| *** gildub_ has quit IRC | 23:51 | |
| clarkb | fwiw I think it should fallbak to master zuul ref if present | 23:51 |
| clarkb | or rather zuul_ref | 23:52 |
| jeblair | clarkb: why? that doesn't make sense to me. | 23:52 |
| clarkb | jeblair: because as in this case we don't test the change | 23:52 |
| jeblair | clarkb: that's because we erroneously told zuul not to test the change | 23:52 |
| jeblair | clarkb: we overrode its default behavior incorrectly | 23:53 |
| dims_ | jeblair : clarkb : stepping out for a bit. | 23:53 |
| clarkb | jeblair: I think it is a bug that you can do that | 23:53 |
| clarkb | zuul should always checkout the change if it exists no? | 23:53 |
| clarkb | if override zuul ref does not exist then checkout zuul ref | 23:53 |
| jeblair | clarkb: no -- because sometimes you just want it to check out the tip of some other branch. that's how upgrade testing works at all. | 23:53 |
| clarkb | if that doens't exist then do branch things as is | 23:53 |
| jeblair | clarkb: it would be really bad to run a grenade upgrade test from master to master, which is what would happen if you could not tell it to check out stable/liberty. | 23:54 |
| clarkb | jeblair: wouldn't upgrade testing have a zuul ref for the earlier branch? | 23:54 |
| jeblair | clarkb: not if there are no changes to that branch | 23:54 |
| jeblair | clarkb: zuul refs only exist if there are changes ahead of the current one that affect that branch | 23:54 |
| clarkb | ok so where it breaks is if you have change for newer branch but don't hvae change for older branch | 23:55 |
| clarkb | because then you check out change for newer branch as if it were on the "old" side | 23:55 |
| *** thorst_ has quit IRC | 23:55 | |
| jeblair | clarkb: i don't believe the checkout logic is broken -- rather, i believe this job does not correctly specify what it wants to do. unfortunately, no one has told me what they want it to do despite my asking. | 23:56 |
| *** ddieterly has joined #openstack-infra | 23:56 | |
| clarkb | jeblair: I mean my proposal is broken in that case | 23:56 |
| jeblair | oh, let me reread then :) | 23:56 |
| clarkb | it would work in all other situations I think but that one and that one is common | 23:56 |
| jeblair | clarkb: right | 23:56 |
| *** rbrndt has quit IRC | 23:56 | |
| jgriffith | jeblair: mordred I propose we set up a test with that type and I can abandon my CI system altogether :) | 23:57 |
| jgriffith | just kidding | 23:57 |
| dims_ | jeblair : i have a review in oslo.config master, i want to check if i release that as a oslo.config library, will i break existing stable/liberty installation that may end up using it (since we don't have caps in g-r) | 23:57 |
| clarkb | my concern with the change as proposed is that it assumes that a change to say stable/mitaka should get master checked out | 23:57 |
| clarkb | rathe rthan checking out stable/mitaka zuul ref for testing against stable/kilo | 23:57 |
| * dims_ running to take his son to a basketball tryout :) | 23:58 | |
| clarkb | dims_: I think you need to set that to the ZUUL_BRANCH instead | 23:58 |
| jeblair | dims_: the fact that you have a stable/liberty branch of oslo.config yet want to test master in the stable/liberty context is mighty confusing. :) | 23:58 |
| clarkb | dims_: and collapse that into the existing job | 23:58 |
| clarkb | jeblair: they want to start doing backward compat again | 23:58 |
| clarkb | jeblair: I think to unravel the stable branch mess for libs | 23:58 |
| clarkb | I will leave comments iwth what I just said on the change as I think that gets us in the correct direction | 23:58 |
| jeblair | clarkb: what's the proposed change? | 23:59 |
| clarkb | jeblair: https://review.openstack.org/#/c/247188/4/jenkins/jobs/devstack-gate.yaml | 23:59 |
| *** maiteb has quit IRC | 23:59 | |
| zaro | clarkb: so i believe the root of the problem behind that '//' bug is that the gerrit redirect no longer works correctly. I plan to revert the apache config on review-dev.o.o back to original and do more testing. | 23:59 |
| clarkb | zaro: ok | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
