| lifeless | adam_g: so hows parallel looking? | 00:01 |
|---|---|---|
| adam_g | lifeless, i only managed to get a few runs thru today but it seems okay? i wonder if the various saucy/mellanox upgrades helped or if it was all just a dream | 00:03 |
| *** Penick has joined #tripleo | 00:03 | |
| adam_g | lifeless, theres an occasional iscsiadm error but it doesnt seem i/o related | 00:03 |
| *** jp_at_hp has quit IRC | 00:04 | |
| *** jp_at_hp1 has joined #tripleo | 00:04 | |
| *** jp_at_hp1 has quit IRC | 00:05 | |
| *** jp_at_hp has joined #tripleo | 00:06 | |
| *** julim has quit IRC | 00:09 | |
| lifeless | adam_g: ok thats great :) | 00:20 |
| *** matsuhashi has joined #tripleo | 00:22 | |
| *** eguz has joined #tripleo | 00:28 | |
| *** eguz has quit IRC | 00:28 | |
| *** jml has quit IRC | 00:28 | |
| *** jdonalds has quit IRC | 00:29 | |
| *** Penick has quit IRC | 00:31 | |
| *** eghobo has quit IRC | 00:33 | |
| *** jml has joined #tripleo | 00:33 | |
| StevenK | Woo, sprint dates posted | 00:35 |
| *** Penick has joined #tripleo | 00:35 | |
| greghaynes | oh? | 00:35 |
| greghaynes | where? | 00:35 |
| *** Penick has quit IRC | 00:36 | |
| StevenK | greghaynes: http://lists.openstack.org/pipermail/openstack-dev/2014-May/036078.html | 00:36 |
| greghaynes | oh crap | 00:37 |
| greghaynes | that might be hard for me to swing | 00:37 |
| greghaynes | but maybe doable | 00:37 |
| *** jtomasek has quit IRC | 00:38 | |
| *** chuckC has quit IRC | 00:38 | |
| StevenK | I'm happy, since it's the weekend after my wifes birthday | 00:38 |
| greghaynes | Im getting married the week after that | 00:39 |
| StevenK | Heh | 00:39 |
| *** yamahata has joined #tripleo | 00:41 | |
| *** chuckC has joined #tripleo | 00:42 | |
| *** ramishra has joined #tripleo | 00:45 | |
| *** chuckC has quit IRC | 00:47 | |
| *** nati_ueno has quit IRC | 00:50 | |
| *** rbrady has joined #tripleo | 00:55 | |
| *** TravT has quit IRC | 00:56 | |
| SpamapS | greghaynes: getting married can wait.. we have a chance to convince people we should use etcd! ;) | 01:02 |
| greghaynes | haha | 01:04 |
| greghaynes | SpamapS: Did you see the etcd discovery protocol deal? | 01:04 |
| greghaynes | Also, Ill totally use that argument when trying to convince the future wife to let me go | 01:05 |
| *** BadCub has joined #tripleo | 01:05 | |
| *** BadCub has quit IRC | 01:09 | |
| SpamapS | greghaynes: You're alerady failing at marriage. You don't convince her to let you go. You convince her that she can't live without you _THEN YOU GO_. ;) | 01:11 |
| clarkb | greghaynes: you better not expect backup from me. Pretty sure if larissa finds out I am supporting you on that I am in more trouble than you would be :P | 01:11 |
| SpamapS | greghaynes: keep in mind, 50% of my marriages have ended in divorce.. so... ;) | 01:11 |
| greghaynes | clarkb: I think this just means you have to come as well | 01:12 |
| clarkb | I can't do anything during that time either. family in town | 01:12 |
| clarkb | s/town/portland/ | 01:12 |
| clarkb | basically end of july early august is worst time for anything in the NW | 01:13 |
| clarkb | because people like you get marriend and family visits :P | 01:13 |
| SpamapS | Hah, it's "the sunny week" | 01:13 |
| clarkb | yup | 01:13 |
| greghaynes | Yep. Gotta make use | 01:13 |
| SpamapS | I don't know how this happened but we somehow jam packed our kids' summer plans to the point where there are literally zero free weeks. Except that one. | 01:14 |
| greghaynes | hahaha | 01:14 |
| *** shakamunyi has quit IRC | 01:16 | |
| clarkb | greghaynes: also all that time in portland is brisket time | 01:17 |
| clarkb | I am addicted | 01:18 |
| SpamapS | Wait | 01:18 |
| SpamapS | a Texan | 01:18 |
| SpamapS | likes... Oregan Brisket? | 01:18 |
| Shrews | Ooh, that location is super convenient for me. Just a 50 minute drive. On the downside, it's 50 min in the worst traffic (except for LA) | 01:18 |
| clarkb | SpamapS: well he was in texas when I made brisket in portland | 01:18 |
| clarkb | SpamapS: but the brisket was pretty damn amazing so I am sure he would have approved | 01:19 |
| SpamapS | ah | 01:19 |
| clarkb | SpamapS: its actually pretty simple. You just need a lot of time and a big hunk of brisket | 01:19 |
| clarkb | we have been doing the barbecue + beer brewing weekends when visiting portland | 01:20 |
| clarkb | works out well because both things take forever and you can multitask relatively well | 01:20 |
| *** nati_ueno has joined #tripleo | 01:21 | |
| *** jp_at_hp has quit IRC | 01:24 | |
| SpamapS | clarkb: I've never cooked a brisket in beer, but I've definitely cooked a brisket all day. :) | 01:26 |
| clarkb | SpamapS: brewing beer is a trap hobby, be careful if you ever start it | 01:28 |
| *** ramishra has quit IRC | 01:28 | |
| clarkb | it wasn't long before I had multiple mash tuns and several kegs and had to keep them full | 01:29 |
| *** nati_ueno has quit IRC | 01:32 | |
| SpamapS | hah no | 01:33 |
| SpamapS | I don't have the patience | 01:34 |
| greghaynes | mmm brisket | 01:34 |
| *** chuckC has joined #tripleo | 01:34 | |
| clarkb | SpamapS: that is why I do it. I need more patience | 01:40 |
| *** nati_ueno has joined #tripleo | 01:42 | |
| *** nati_ueno has quit IRC | 01:42 | |
| *** yamahata has quit IRC | 01:43 | |
| *** yamahata has joined #tripleo | 01:43 | |
| *** nosnos has joined #tripleo | 01:45 | |
| SpamapS | clarkb: need shmeed.. patience is for monks. I need more money. And beer. And sleep. Patience won't get me any of those things. ;) | 01:46 |
| clarkb | SpamapS: well it can help on the more beer front. When was the last time you had 15 gallon of beer at hmoe for yourself? | 01:47 |
| *** ramishra has joined #tripleo | 01:48 | |
| TheJulia | SpamapS: I have to agree, use patience to make beer, sell bear to friends, have more money. Its how brewerys are born. | 01:49 |
| TheJulia | err, beer | 01:50 |
| SpamapS | If I can make a BEAR with patience, I'm in. | 01:50 |
| SpamapS | who doesn't want a bear? | 01:50 |
| clarkb | also apparently there is a 200gallon/yr home brew limit | 01:50 |
| clarkb | I haven't had enough patience to break that law yet | 01:50 |
| SpamapS | and anybody with beer will give me their beer if I have a bear. | 01:50 |
| pleia2 | bears++ | 01:51 |
| TheJulia | lol | 01:51 |
| SpamapS | ok.. breakfast time | 01:51 |
| TheJulia | and bedtime... | 01:51 |
| StevenK | Haha | 01:52 |
| *** weshay has joined #tripleo | 01:58 | |
| *** weshay has quit IRC | 02:00 | |
| *** noslzzp has quit IRC | 02:13 | |
| *** jdonalds has joined #tripleo | 02:14 | |
| lifeless | o/ | 02:16 |
| rbrady | SpamapS: anyone who tells you they brew beer as a hobby is lying. It's 90% cleaning and sanitizing. | 02:16 |
| clarkb | rbrady: it depends on how seriously you take perfection | 02:18 |
| clarkb | you don't need to do all that much cleaning and sanitizing if you don't sweat it a whole lot | 02:18 |
| rbrady | clarkb: I've recently switched to using RO water and dosing it with ions to meet the profile I want to brew with | 02:18 |
| clarkb | rbrady: that is far too much work :P but I am also lucky enough to have access to portland water when I brew | 02:19 |
| SpamapS | clarkb: cool, so just enough e.coli and drunk-guy-pee to make it distinct and nutty? | 02:20 |
| clarkb | SpamapS: exactly | 02:20 |
| * SpamapS heads to breakfast for reals | 02:20 | |
| clarkb | its actually tigard water which didn't have the e.coli or drunk guy | 02:20 |
| clarkb | but is basically the same stuff | 02:20 |
| rbrady | clarkb: https://plus.google.com/u/0/photos/117124471514241857344/albums/6011840662489241201?authkey=CJviwou10tKFuwE | 02:21 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/diskimage-builder: Add new cloud-init-datasources element https://review.openstack.org/95598 | 02:22 |
| greghaynes | Thats why we make beer out of our water | 02:22 |
| greghaynes | its no good for drinking | 02:22 |
| clarkb | rbrady: nice. I bet that pump means you have a lot more cleaning than me | 02:22 |
| clarkb | I am cheap and use muscle + gravity | 02:22 |
| rbrady | clarkb: actually, I collect a keg of runoff from the first few minutes of the wort chiller and then run that back through the pump at the end | 02:23 |
| clarkb | oh smart | 02:24 |
| clarkb | if I end up in a place with enough room again I will think about upgrading equipment. As is though its all at my brothers and I brew when I can | 02:25 |
| clarkb | so its all cheap. 10gallon igloo mash tun with 10 gallon steel pot for boiling and only one burner :) | 02:27 |
| rbrady | I enjoy the differences in equipment setups | 02:27 |
| rbrady | there's almost never two alike | 02:27 |
| rbrady | I tried single vessel BIAB for a bit, but it's awkward to manage lifting out of the kettle for 10 gal batches | 02:29 |
| rbrady | and squeezing the bag made horrible tannin flavored beer | 02:29 |
| clarkb | I also tend to use the electric stove to heat strike and sparge water because I can do about 4 gallons ok | 02:30 |
| clarkb | then move to propane for the boil | 02:31 |
| clarkb | this is super handy in the winter when I dno't want to be in the cold outside all day :) | 02:31 |
| SpamapS | nice.. hotel wifi is sane enough that going from room to restaurant does not drop active SSH connections. | 02:34 |
| StevenK | Heh, nice | 02:35 |
| StevenK | I think I finally have enough context to start docs building for os-cloud-config | 02:36 |
| StevenK | The rest of os-*-config need more work | 02:36 |
| StevenK | Like actual docs | 02:36 |
| *** jdonalds has quit IRC | 02:39 | |
| *** edmund has joined #tripleo | 02:51 | |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1321943 | 03:00 |
| uvirtbot | Launchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1318767 | 03:00 |
| uvirtbot | Launchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1314978 | 03:00 |
| uvirtbot | Launchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1315355 | 03:00 |
| uvirtbot | Launchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1323167 | 03:00 |
| uvirtbot | Launchpad bug 1323167 in tripleo "overcloud can't create an instance by neutron error" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1319473 | 03:00 |
| uvirtbot | Launchpad bug 1319473 in tripleo "devtest_testenv.sh doesn't honour overcloud_computescale or overcloud_controlscale" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1316675 | 03:00 |
| uvirtbot | Launchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 03:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| *** matsuhashi has quit IRC | 03:00 | |
| *** matsuhashi has joined #tripleo | 03:04 | |
| *** BadCub has joined #tripleo | 03:06 | |
| *** BadCub has quit IRC | 03:10 | |
| *** ramishra has quit IRC | 03:12 | |
| *** nosnos has quit IRC | 03:33 | |
| *** matsuhashi has quit IRC | 03:34 | |
| *** matsuhashi has joined #tripleo | 03:35 | |
| *** matsuhashi has quit IRC | 03:39 | |
| *** eghobo has joined #tripleo | 03:46 | |
| *** shakamunyi has joined #tripleo | 03:48 | |
| *** lazy_prince has joined #tripleo | 03:54 | |
| *** edmund has quit IRC | 03:55 | |
| *** matsuhashi has joined #tripleo | 03:56 | |
| *** shakamunyi has quit IRC | 04:05 | |
| *** rpodolyaka1 has joined #tripleo | 04:06 | |
| jogo | looks like gate is failing http://jogo.github.io/gate/tripleo.html | 04:06 |
| greghaynes | Kiall: does https://review.openstack.org/#/c/93032/ need to depend on https://review.openstack.org/#/c/93039 ? I would really like to merge 93032 and 93039 seems a lot less straightforward... | 04:07 |
| lifeless | jogo: ugh | 04:08 |
| lifeless | greghaynes: you could do a test run yourself and if ok rebase it | 04:09 |
| * greghaynes does this | 04:09 | |
| jogo | lifeless: you probably can make a bot do something in IRC every time graphite shows an outage | 04:09 |
| greghaynes | I just saw a pretty bad mysql fail in a different test... hopefully this fixes | 04:10 |
| *** akuznetsov has joined #tripleo | 04:11 | |
| StevenK | lifeless: https://review.openstack.org/95445 would help testing locally, and stop the nova-bm ISEs I get once and then have to patch locally | 04:12 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Clamp pm_password to 255 for nova-bm nodes https://review.openstack.org/95445 | 04:14 |
| *** vinsh has quit IRC | 04:21 | |
| *** jdonalds has joined #tripleo | 04:31 | |
| *** lazy_prince has quit IRC | 04:32 | |
| *** ramishra has joined #tripleo | 04:36 | |
| lifeless | greghaynes: whee https://review.openstack.org/#/c/86435/ was unhappy | 04:37 |
| lifeless | I mean, I expected it without the heat change | 04:37 |
| lifeless | but wheee | 04:37 |
| lifeless | also HTF did f20 work | 04:37 |
| lifeless | I guess null==null | 04:37 |
| *** jdonalds has quit IRC | 04:40 | |
| lifeless | jogo: so | 04:43 |
| lifeless | jogo: searching for "Error communicating with http://192.0.2.1:6385/ timed out" | 04:43 |
| lifeless | finds nothing | 04:43 |
| lifeless | but | 04:43 |
| *** Penick has joined #tripleo | 04:43 | |
| lifeless | oh,time window. headdesk | 04:43 |
| *** Penick has quit IRC | 04:43 | |
| * lifeless tries a longer window | 04:44 | |
| lifeless | nope | 04:44 |
| lifeless | 7 days, its not found | 04:44 |
| greghaynes | wow that failed fast | 04:44 |
| lifeless | http://logstash.openstack.org/#eyJzZWFyY2giOiJcIkVycm9yIGNvbW11bmljYXRpbmcgd2l0aCBodHRwOi8vMTkyLjAuMi4xOjYzODUvIHRpbWVkIG91dFwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDAxMzM4NjIwMTgwfQ== | 04:44 |
| jogo | lifeless: sample console log you are looking at? | 04:44 |
| lifeless | jogo: http://logs.openstack.org/76/84076/7/check-tripleo/check-tripleo-overcloud-precise/74eb911/console.html | 04:45 |
| *** nosnos has joined #tripleo | 04:45 | |
| lifeless | erm | 04:45 |
| lifeless | not that one, sec | 04:45 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Replace nagios3 template with HOT rewrite https://review.openstack.org/90665 | 04:46 |
| lifeless | ETOOMANYTABS | 04:46 |
| lifeless | jogo: http://logs.openstack.org/33/93133/41/check-tripleo/check-tripleo-ironic-seed-precise/07c7a26/console.html | 04:46 |
| lifeless | jogo: from https://review.openstack.org/#/c/93133/ | 04:46 |
| jogo | ohh ithinktry message:"Error communicating with <a href=\'http://192.0.2.1:6385/\'>http://192.0.2.1:6385/</a> timed out" | 04:46 |
| jogo | that appears to be a unique failure | 04:47 |
| lifeless | brilliant | 04:47 |
| lifeless | thanks | 04:47 |
| lifeless | one-off FTW :( | 04:48 |
| jogo | np | 04:48 |
| jogo | I found that by searching for error communicating with | 04:48 |
| jogo | as you see links are special sometimes | 04:48 |
| lifeless | jogo: I see | 04:48 |
| jogo | html and all | 04:48 |
| lifeless | yes, stab. | 04:48 |
| lifeless | recheck noooooo bug | 04:48 |
| jogo | ohh 3 more hits from last week | 04:50 |
| jogo | message:"Error communicating with" AND message:"timed out" | 04:50 |
| lifeless | worth making an elastic recheck bug rule ? | 04:50 |
| jogo | don't see why not! | 04:51 |
| jogo | use message:"Error communicating with" AND message:"timed out" AND build_queue:"check-tripleo" | 04:51 |
| lifeless | let me file a bug | 04:51 |
| lifeless | were all four on the ironic job ? | 04:52 |
| jogo | or build_queue:"check-tripleo" AND message:"Error communicating with <a href=\'http://192.0.2.1:6385/\'>http://192.0.2.1:6385/</a> timed out" | 04:52 |
| lifeless | whats the repo to patch to add rules? | 04:52 |
| jogo | yes, you can click on build_name in kibana | 04:52 |
| jogo | git.openstack.org/cgit/openstack-infra/elastic-recheck/ | 04:52 |
| *** jdonalds has joined #tripleo | 04:53 | |
| lifeless | is there a standard tag for recheck bugs? | 04:54 |
| lifeless | I know this is documented, sorry for bootstrapping myself on you | 04:54 |
| lifeless | jogo: ^ | 04:54 |
| *** eghobo has quit IRC | 04:57 | |
| *** eghobo has joined #tripleo | 04:57 | |
| jogo | no tag actually | 04:57 |
| *** jdonalds has quit IRC | 04:59 | |
| *** eguz has joined #tripleo | 04:59 | |
| lifeless | this may be a symptom of neutron unreliability in the tripleo hp1 region | 04:59 |
| lifeless | nope | 04:59 |
| lifeless | tripleo-precise-tripleo-test-cloud-rh1-4402332 | 04:59 |
| *** jdonalds has joined #tripleo | 05:02 | |
| lifeless | all on the rh cloud | 05:02 |
| lifeless | not that a sample size of 3 is much | 05:02 |
| *** shakamunyi has joined #tripleo | 05:02 | |
| *** eghobo has quit IRC | 05:03 | |
| *** lazy_prince has joined #tripleo | 05:05 | |
| lifeless | jogo: whats the landing process for recheck rules? 1 core? 2? does it get CI ? | 05:06 |
| *** jdonalds has quit IRC | 05:06 | |
| *** BadCub has joined #tripleo | 05:07 | |
| *** ramishra has quit IRC | 05:10 | |
| *** BadCub has quit IRC | 05:11 | |
| *** ramishra has joined #tripleo | 05:12 | |
| *** shakamunyi has quit IRC | 05:14 | |
| jogo | lifeless: http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/README.rst#n25 should answer most of your questsions | 05:16 |
| jogo | but 1 core and it has some CI but usually manual inspection of the query | 05:16 |
| *** ramishra has quit IRC | 05:16 | |
| lifeless | jogo: ok, so you can land my new one ? :) | 05:17 |
| lifeless | get me hooked, I want my fix man | 05:18 |
| jogo | its nice to use the Relatd-Bug: tag | 05:24 |
| jogo | in the commit msg so the bug gets a link to the patch | 05:24 |
| jogo | but that is easy enough to do manually | 05:25 |
| jogo | being this is one of the first tripleo patches I want to get a second core on it | 05:25 |
| lifeless | roger | 05:26 |
| lifeless | I can repush with a tag if you want | 05:26 |
| jogo | yeah that would be good, also maybe mention in the commit msg that this is for tripleo | 05:27 |
| jogo | make it entertaining | 05:27 |
| *** rpodolyaka1 has quit IRC | 05:29 | |
| lazy_prince | Revivew please for https://review.openstack.org/#/c/95655 | 05:30 |
| lifeless | man spec review is non-trivial | 05:30 |
| lifeless | I have to like think | 05:30 |
| lifeless | jogo: done | 05:30 |
| *** dshulyak_ has joined #tripleo | 05:31 | |
| lifeless | lazy_prince: thank you | 05:31 |
| lazy_prince | lifeless: my pleasure.. is there anything else that I can pick up..? | 05:32 |
| lifeless | always ;) | 05:32 |
| lifeless | have a look through the bug tracker or possibly (its not really gardened yet) the blueprints | 05:33 |
| lazy_prince | sure.. I will take a look.. | 05:33 |
| lifeless | jogo: splitting http://jogo.github.io/gate/tripleo.html out by region would be super useful | 05:33 |
| lifeless | the host name has rh1 for red hat 1 | 05:33 |
| lifeless | and and just tripleo-test-cloud for hp1. | 05:34 |
| lifeless | I'll submit a change about that while I'm thinking of it | 05:34 |
| jogo | that data comes from graphite which doesn't know about regions | 05:35 |
| jogo | at least I don't know how to find regions in graphite | 05:35 |
| lifeless | jogo: so we'd need to tag it on input ? | 05:35 |
| lifeless | providers if its coming from nodepool/jenkins | 05:35 |
| jogo | so logstash has that data I think | 05:36 |
| jogo | let me dig a bit | 05:36 |
| lifeless | https://review.openstack.org/96354 may affect that if/when it lands | 05:36 |
| *** tzumainn has quit IRC | 05:38 | |
| *** shakamunyi has joined #tripleo | 05:40 | |
| jogo | I need to head out but: | 05:43 |
| jogo | build_queue:"check-tripleo" AND (message:"Finished: FAILURE") AND build_node:*rh1* | 05:43 |
| jogo | for RH failures | 05:43 |
| jogo | for not rh1 failures" build_queue:"check-tripleo" AND (message:"Finished: FAILURE") AND NOT build_node:*rh1* | 05:43 |
| lazy_prince | lifeless: can you also review https://review.openstack.org/#/c/79873/ and give me a +2 (possibly) | 05:43 |
| jogo | build_queue:"check-tripleo" AND (message:"Finished: FAILURE" OR message:"Finished: SUCCESS") AND NOT build_node:*rh1* | 05:44 |
| jogo | ^ total number of jobs | 05:44 |
| * jogo looks at the clock and wonders off | 05:44 | |
| *** shakamunyi has quit IRC | 05:50 | |
| *** rpodolyaka1 has joined #tripleo | 05:56 | |
| *** rpodolyaka1 has quit IRC | 06:02 | |
| tchaypo | lifeless: so how important is pycon-au to us? | 06:05 |
| tchaypo | because the proposed dates for the mid-cycle complete knock it out | 06:05 |
| *** morganfainberg is now known as stevemar2 | 06:09 | |
| *** stevemar2 is now known as morganfainberg | 06:10 | |
| tchaypo | tchaypo: lifeless: as in, we'd have to leave NC on the 4:40pm flight on the 30th, after just 2.5 days of the mid-cycle, to arrive back in BNE by 7am on the day of the miniconf (at least if we're flying delta, I haven't checked for other airlines but I don't think they'll make much difference) | 06:12 |
| lifeless | I need to reply to jaromir and say I can't make it ;) | 06:16 |
| *** rdopiera has quit IRC | 06:17 | |
| *** rdopiera has joined #tripleo | 06:17 | |
| *** rdopiera has quit IRC | 06:18 | |
| *** rdopiera has joined #tripleo | 06:18 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Tests are broken since Horizon started using angular-cookies https://review.openstack.org/96152 | 06:20 |
| *** jprovazn has joined #tripleo | 06:22 | |
| * tchaypo goes back to be sick in bed | 06:23 | |
| lifeless | tchaypo: sadface | 06:24 |
| *** dshulyak_ has quit IRC | 06:26 | |
| zigo | lifeless: Hi there! | 06:29 |
| lifeless | zigo: ol | 06:30 |
| lifeless | zigo: ola | 06:30 |
| zigo | I'm looking at the todo you gave me in Atlanta. | 06:30 |
| zigo | 1/ Modify tripleo incubator to use packages instead of using git clone | 06:30 |
| zigo | :) | 06:30 |
| zigo | How / where do I make this optional ? | 06:31 |
| zigo | lifeless: ^ | 06:31 |
| *** jtomasek has joined #tripleo | 06:33 | |
| zigo | Also, should tripleo-incubator be packaged in Debian? | 06:33 |
| *** lazy_prince has quit IRC | 06:38 | |
| SpamapS | zigo: definitely not | 06:39 |
| *** lazy_prince has joined #tripleo | 06:40 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Improve percona cluster initialization https://review.openstack.org/93032 | 06:40 |
| lifeless | zigo: Its not configurable at the moment; but we generally only depend on things being in the path | 06:42 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Improve percona cluster initialization https://review.openstack.org/93032 | 06:42 |
| *** rakesh_hs has joined #tripleo | 06:42 | |
| lifeless | zigo: so I'd say add an option to say to use packages, and have the option disable the cloning of tools | 06:42 |
| lifeless | zigo: if you want more specific guidance, I suggest putting a tripleo-specs patch up about it and we can work through and capture all the cases there | 06:43 |
| zigo | lifeless: And where do I put such option? | 06:43 |
| zigo | There's no etc/tripleo-incubator, is there? | 06:43 |
| lifeless | nope | 06:43 |
| lifeless | I meant command line option :) | 06:43 |
| zigo | Oh ok! :) | 06:44 |
| zigo | Sorry, I got no clue how all this works! :/ | 06:44 |
| lifeless | mordred: yo ping | 06:44 |
| greghaynes | Kiall: Hope you dont mind, made some changes to https://review.openstack.org/#/c/93032 so we can try and get it merged - seeing some test failures which I think need this | 06:44 |
| zigo | lifeless: So, I'm guessing it's one of the scripts in "scripts/" that you were talking about, right? Which one should I look into? | 06:46 |
| *** shakamunyi has joined #tripleo | 06:47 | |
| lifeless | zigo: I think you probably need to start with devtest.sh and read and read | 06:47 |
| lifeless | zigo: doing this will require getting your head around how it all works - its not a huge amount of code | 06:47 |
| zigo | Thanks for the clues! Will read that script. | 06:47 |
| lifeless | obviously do ask questions if you hit the wall | 06:47 |
| zigo | Sure. | 06:47 |
| zigo | :) | 06:47 |
| zigo | This really helps, because I needed a starting point. | 06:49 |
| lifeless | Yeah, some thread to pull on :) - I know the feeling | 06:49 |
| *** lokesh184 has joined #tripleo | 06:53 | |
| xuhaiwei | I wanted to add a new compute node to the overcloud, so i updated the overcloud.yaml, and use heat stack-update, but update is failed, i can't use heat stack-update again, so how to add the compute node? | 06:54 |
| *** shakamunyi has quit IRC | 06:59 | |
| *** xuhaiwei_ has joined #tripleo | 07:01 | |
| *** xuhaiwei has quit IRC | 07:01 | |
| *** jdonalds has joined #tripleo | 07:03 | |
| xuhaiwei_ | lifeless: hi, about yesterday's question, I didn't succeed in doing virsh console undercloud, I got an error and didn't solve it. but really strange, I can ssh undercloud today | 07:04 |
| xuhaiwei_ | lifeless: but still can't ssh overcloud | 07:04 |
| mordred | lifeless: pong | 07:08 |
| GheRivero | morning TripleO! | 07:10 |
| jprovazn | greghaynes: Kiall: Hi, today's updates of https://review.openstack.org/#/c/93032/ (patchsets 9+10) included backup of innodb files - this is already addressed in other patch waiting for more reviews: https://review.openstack.org/#/c/93104/ (+ https://review.openstack.org/#/c/86871/) | 07:10 |
| greghaynes | hrm | 07:11 |
| greghaynes | jprovazn: So the restart part of the reset-db is really an issue for galera | 07:19 |
| greghaynes | or any clustered setup (as mentioned in your commit msg) | 07:19 |
| greghaynes | This makes me think we should just clear the mysql data before starting it the first time since we are running mysql from o-r-c anyways | 07:20 |
| greghaynes | then restarting isnt an issue | 07:20 |
| jprovazn | mmt | 07:20 |
| jprovazn | greghaynes: maybe you mean this part? https://review.openstack.org/#/c/86871/3/elements/mysql/os-refresh-config/configure.d/51-mysql-init | 07:21 |
| jprovazn | restart was removed and is done in https://review.openstack.org/#/c/86871/3/elements/mysql/os-refresh-config/configure.d/53-init-mysql-cluster | 07:22 |
| greghaynes | oh I see, you did it exactly opposite of how I did | 07:22 |
| *** pblaho has joined #tripleo | 07:22 | |
| *** eguz has quit IRC | 07:22 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-heat-templates: Haproxy configuration https://review.openstack.org/93554 | 07:23 |
| jprovazn | really? I should recheck Kiall's patch, but I thought that it backups innodb files before cluster init - only difference is just that in Kiall's patch it's part of the same script | 07:23 |
| jprovazn | + a check if it's bootstrap or not which I think is not needed | 07:23 |
| greghaynes | I think in yours it does db init then reset-mysql-files which causes restart of mysql? | 07:24 |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-heat-templates: Haproxy configuration https://review.openstack.org/93554 | 07:24 |
| jprovazn | the problem of having this in same script is that the reset-db part is has to be done for mariadb too | 07:24 |
| greghaynes | Are you sure? Im actually unconvinced that process is needed at all | 07:24 |
| *** shakamunyi has joined #tripleo | 07:25 | |
| *** bogdando has quit IRC | 07:26 | |
| * greghaynes runs percona build without the file backup part | 07:26 | |
| *** bogdando has joined #tripleo | 07:26 | |
| jprovazn | greghaynes: well, it might be that it works w/o the cleanup part too - I suppose dprince added it for some reason | 07:27 |
| jprovazn | greghaynes: it might be just because of save some space | 07:28 |
| jprovazn | greghaynes: actually SpamapS added this - 7341771632ca0a67c2a559608812e7360d63c174 | 07:30 |
| greghaynes | hrm, ok | 07:30 |
| greghaynes | I think were going to switch back to packages at some point so that might not be a bad idea... | 07:31 |
| *** jp_at_hp has joined #tripleo | 07:31 | |
| *** ifarkas has joined #tripleo | 07:31 | |
| greghaynes | jprovazn: In https://review.openstack.org/#/c/93104/ I think its a better idea to do the cleanup before 51-mysql-init so the mysql init element can start up mysql however is needed | 07:33 |
| greghaynes | im thinking in the case of a clustered setup you cant simply call restart and expect it to work - different nodes need to perform different operations | 07:33 |
| greghaynes | and that logic would need to live in each specific mysql element | 07:34 |
| *** jistr has joined #tripleo | 07:34 | |
| jprovazn | greghaynes: this patch only extracts "reset files" logic out of reset-db script, mysql restart is taken care of the dependency patch | 07:35 |
| *** shakamunyi has quit IRC | 07:36 | |
| greghaynes | Yes, the real reason for my needing to move that portion of reset-db was that the os-svc-restart mysql cant be expected to work | 07:37 |
| jprovazn | greghaynes: then here: https://review.openstack.org/#/c/86871/3/elements/mysql-common/os-refresh-config/configure.d/52-reset-mysql-files restart is removed and is expected to be done in *separate* script | 07:37 |
| jprovazn | greghaynes: so then 52-xxx script backups files and 53-xxx script (different for each mysql flavor) restarts cluster | 07:38 |
| jprovazn | https://review.openstack.org/#/c/86871/3/elements/mysql/os-refresh-config/configure.d/53-init-mysql-cluster | 07:38 |
| greghaynes | That seems to by trying to solve the restart problem in general, which is a lot harder and different - this is just an issue with being able to do a single boot up which id like to solve sooner | 07:40 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Choose whether to deploy/update using heat. https://review.openstack.org/92344 | 07:41 |
| jprovazn | greghaynes: to be sure, " single boot up" == the issue of "mysql restart" being part of reset-db script? | 07:41 |
| *** pelix has joined #tripleo | 07:41 | |
| greghaynes | Yes, that is causing random test failures for me right now | 07:42 |
| jprovazn | greghaynes: "solve the restart problem in general" - you mean because https://review.openstack.org/#/c/86871/ includes cehck if a node is already in cluster? | 07:43 |
| jprovazn | greghaynes: the issue with mysql restart in reset-db is addressed in the patch I linked above | 07:44 |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-incubator: updates work behind proxy Closes Bug:1324398 https://review.openstack.org/96383 | 07:45 |
| jprovazn | greghaynes: this patch doesn't solve restart problem in general - it adds just a very minimum logic to be able t ostart more than one node | 07:45 |
| jprovazn | greghaynes: w/o this the existing code would break at all (you would start bootstrap node all the time as standalone) | 07:46 |
| jprovazn | greghaynes: restart problem in general - I expect this would be solved by adding a more sophisticated tool described in your proposal: https://review.openstack.org/#/c/94739/2/specs/juno/leader-election.rst | 07:47 |
| greghaynes | jprovazn: Left a comment on https://review.openstack.org/#/c/86871/ | 07:48 |
| greghaynes | The issue is I dont think that what that patch tries to do is solveable without that spec | 07:49 |
| greghaynes | unless im misunderstanding what bootstrap-pxc does... | 07:50 |
| *** lazy_prince has quit IRC | 07:50 | |
| greghaynes | As is, it was intentional that restarting doesnt work because theres a possibility of data loss | 07:51 |
| jprovazn | greghaynes: jsut to be clear - what do you mean by restarting? restarting of whole cluster or a single node? | 07:52 |
| *** martyntaylor has joined #tripleo | 07:52 | |
| *** lazy_prince has joined #tripleo | 07:53 | |
| greghaynes | What you do when you start and cannot find a cluster | 07:53 |
| jprovazn | greghaynes: oooh, you mean the "|| service mysql bootstrap-pxc"? | 07:54 |
| greghaynes | yes | 07:54 |
| jprovazn | greghaynes: ok, no problem to remove this and just fail if a node reconnects and none of other nodes is reachable | 07:54 |
| *** morganfainberg is now known as morganfainberg_Z | 07:54 | |
| greghaynes | Yes, but in that case it seems nearly equivilent to our db.initialized lock | 07:56 |
| *** matsuhashi has quit IRC | 07:58 | |
| jprovazn | greghaynes: actually not - what you do in this script is that: | 07:59 |
| *** matsuhashi has joined #tripleo | 07:59 | |
| jprovazn | hm, mmt | 07:59 |
| greghaynes | so backing up a bit - my issue with https://review.openstack.org/#/c/93104 is why not do the db file clearing before 51-mysql | 08:01 |
| jprovazn | greghaynes: you don't handle any restart at all (including metadata update), with this patch https://review.openstack.org/#/c/86871/ restarting of mysql service works, what doesn't work would be restarting *whole* cluster | 08:02 |
| *** matsuhas_ has joined #tripleo | 08:02 | |
| *** shakamunyi has joined #tripleo | 08:03 | |
| *** matsuhashi has quit IRC | 08:04 | |
| greghaynes | the problem is on the bootstrap node, if 51-mysql-init starts mysql, then reset-db restarts it we end up in the state where we dont come back | 08:04 |
| greghaynes | since we are the only node in the cluster at that point (usually) | 08:05 |
| jprovazn | greghaynes: the patch https://review.openstack.org/#/c/93104 does not solve restarting mysql (behavior is same), it just extracts thsi logic into separate script so mysql restart is then easier in next patch | 08:08 |
| *** e0ne has joined #tripleo | 08:12 | |
| *** morganfainberg_Z is now known as morganfainberg | 08:12 | |
| *** shakamunyi has quit IRC | 08:14 | |
| greghaynes | jprovazn: Commented | 08:14 |
| greghaynes | jprovazn: That would let me rebase my change on yours and remove my reset-db changes from mine | 08:15 |
| *** lucasagomes has joined #tripleo | 08:16 | |
| greghaynes | basically as is it does start mysql -> clean up files -> restart. I was hoping to change it to clean up files -> start mysql once | 08:17 |
| jprovazn | greghaynes: well, this exactly does the patch referenced in "needed by" of https://review.openstack.org/#/c/93104 | 08:20 |
| jprovazn | greghaynes: I think that the problem is different opinion on https://review.openstack.org/#/c/93104 | 08:20 |
| jprovazn | because I think it's not sufficient to do "mysql restart" as is done now, nor ignore mysql restart when mysql was started once | 08:21 |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-image-elements: Prevent duplication of net_binds per service https://review.openstack.org/95097 | 08:22 |
| jprovazn | and because restarting mysql node is more complex (even in naive solution) it's a separate patch https://review.openstack.org/#/c/86871/ | 08:22 |
| jprovazn | greghaynes: I can probably remove that restart to get some patches in | 08:25 |
| *** derekh_ has joined #tripleo | 08:25 | |
| jprovazn | but it really doesn't solve the situation until https://review.openstack.org/#/c/86871/ is merged | 08:25 |
| greghaynes | agreed. Im just afraid 86871 is a big step in complexity vs. a simple but less than ideal solution | 08:26 |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-incubator: Allow user-specified timeout for stack creation https://review.openstack.org/95968 | 08:26 |
| greghaynes | or maybe I just need to get to bed and am not seeing clearly ;) | 08:28 |
| jp_at_hp | lifeless: thanks for the comments on https://review.openstack.org/#/c/93597/ - I think you are agreeing that there is a valid bug to be fixed there... | 08:30 |
| jprovazn | greghaynes: well, if I should choose between the script vs state where mysql is not being restarted on any changes, I would choose the script ;) | 08:31 |
| *** xuhaiwei_ has quit IRC | 08:31 | |
| lifeless | jp_at_hp: :) | 08:33 |
| jprovazn | greghaynes: ouch, https://review.openstack.org/#/c/86435/12 require another update based on lifeless' patch | 08:39 |
| *** shakamunyi has joined #tripleo | 08:40 | |
| *** jcoufal has joined #tripleo | 08:46 | |
| lifeless | jcoufal: o/ | 08:50 |
| jcoufal | lifeless: hey | 08:51 |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-incubator: updates work behind proxy adding -E for apt-get and yum Closes Bug:1324398 https://review.openstack.org/96383 | 08:51 |
| *** shakamunyi has quit IRC | 08:52 | |
| lifeless | jcoufal: I've mailed the list - but is there any room to wiggle those dates? | 08:53 |
| *** matsuhas_ has quit IRC | 08:53 | |
| jcoufal | lifeless: there might be, I'll gather all feedback and we will see what we can do, I'll respond on the list | 08:55 |
| lifeless | jcoufal: cool, thanks! | 08:56 |
| *** matsuhashi has joined #tripleo | 08:58 | |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1321943 | 09:00 |
| uvirtbot | Launchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1318767 | 09:00 |
| uvirtbot | Launchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1324398 | 09:00 |
| uvirtbot | Launchpad bug 1324398 in tripleo "proxy is not used when tripleo incubator runs install dependencies." [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1314978 | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1315355 | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1323167 | 09:00 |
| uvirtbot | Launchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1319473 | 09:00 |
| uvirtbot | Launchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1316675 | 09:00 |
| uvirtbot | Launchpad bug 1323167 in tripleo "overcloud can't create an instance by neutron error" [Undecided,New] | 09:00 |
| uvirtbot | Launchpad bug 1319473 in tripleo "devtest_testenv.sh doesn't honour overcloud_computescale or overcloud_controlscale" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 09:00 |
| uvirtbot | Launchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 09:00 |
| lifeless | wall | 09:06 |
| lifeless | of | 09:06 |
| lifeless | bugs | 09:06 |
| rdopiera | in Soviet Russia bugs report YOU | 09:07 |
| lifeless | jprovazn: I bet greghaynes is asleep now | 09:08 |
| lifeless | jprovazn: if so, consider fixing up and pushing yourself ? | 09:08 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Tests are broken since Horizon started using angular-cookies https://review.openstack.org/96152 | 09:12 |
| openstackgerrit | Om Kumar proposed a change to openstack/diskimage-builder: Refactor code to select boot kernel https://review.openstack.org/79873 | 09:15 |
| StevenK | I think that knocks half of the bugs out of untriaged | 09:16 |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-specs: Haproxy configuration options https://review.openstack.org/94907 | 09:17 |
| *** shakamunyi has joined #tripleo | 09:19 | |
| *** e0ne has quit IRC | 09:20 | |
| *** e0ne has joined #tripleo | 09:21 | |
| *** dguerri is now known as _dguerri | 09:22 | |
| *** _dguerri is now known as dguerri | 09:26 | |
| openstackgerrit | Jiri Stransky proposed a change to openstack/os-cloud-config: PKI key/cert generation for use with Keystone https://review.openstack.org/83324 | 09:27 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Add os-is-bootstrap-host element and script https://review.openstack.org/86435 | 09:31 |
| *** shakamunyi has quit IRC | 09:31 | |
| *** matsuhashi has quit IRC | 09:33 | |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-incubator: added -E for zypper updates work behind proxy adding -E for apt-get and yum Closes Bug:1324398 https://review.openstack.org/96383 | 09:33 |
| *** matsuhashi has joined #tripleo | 09:33 | |
| *** matsuhashi has quit IRC | 09:35 | |
| *** matsuhashi has joined #tripleo | 09:36 | |
| *** lynxman has quit IRC | 09:36 | |
| *** ccorrigan has joined #tripleo | 09:37 | |
| *** lynxman has joined #tripleo | 09:37 | |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-incubator: Make install-dependencies use env proxy Closes Bug:1324398 https://review.openstack.org/96383 | 09:38 |
| lsmola | slagle: hello, do you think you can checkout this one https://review.openstack.org/#/c/94634/ ? | 09:43 |
| dshulyak | jrpovazn: https://review.openstack.org/#/c/86435/14/elements/seed-stack-config/config.json | 09:46 |
| openstackgerrit | Ethan Lynn proposed a change to openstack/diskimage-builder: Add Ubuntu local image support https://review.openstack.org/90887 | 09:49 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Add os-is-bootstrap-host element and script https://review.openstack.org/86435 | 09:49 |
| jprovazn | dshulyak: lifeless: ^ updated | 09:50 |
| lxsli | slagle: what's up with https://review.openstack.org/#/c/92400/ please? | 09:52 |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-incubator: Make install-dependencies use env proxy https://review.openstack.org/96383 | 09:53 |
| *** viktors|afk is now known as viktors | 09:55 | |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-incubator: Make install-dependencies use env proxy https://review.openstack.org/96383 | 09:57 |
| *** shakamunyi has joined #tripleo | 09:58 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Adding missing setup for instack https://review.openstack.org/91078 | 09:59 |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-incubator: Make install-dependencies use env proxy https://review.openstack.org/96383 | 10:02 |
| *** nosnos has quit IRC | 10:02 | |
| *** jtomasek has quit IRC | 10:03 | |
| *** nosnos has joined #tripleo | 10:05 | |
| *** shakamunyi has quit IRC | 10:05 | |
| *** shakamunyi has joined #tripleo | 10:05 | |
| *** jtomasek has joined #tripleo | 10:05 | |
| *** jml has quit IRC | 10:10 | |
| *** morazi has quit IRC | 10:10 | |
| *** rcarrillocruz has quit IRC | 10:10 | |
| *** jesusaurus has quit IRC | 10:10 | |
| *** rcarrillocruz has joined #tripleo | 10:10 | |
| *** shakamunyi has quit IRC | 10:10 | |
| *** jml has joined #tripleo | 10:10 | |
| *** morazi has joined #tripleo | 10:10 | |
| *** shakamunyi has joined #tripleo | 10:10 | |
| *** jesusaurus has joined #tripleo | 10:11 | |
| lxsli | Cores, please could I get eyes on tripleo-spec: https://review.openstack.org/#/c/94910 | 10:12 |
| lxsli | lifeless seems happy with it but I'd like it to get more exposure | 10:12 |
| *** shakamunyi has quit IRC | 10:20 | |
| SpamapS | lxsli: I think I'm too jet lagged to fully understand. :-P | 10:22 |
| SpamapS | I tried :-/ | 10:22 |
| lxsli | SpamapS: OK thanks, if you could find a few mins in the next couple of days to try again that'd be cool :) | 10:24 |
| ccorrigan | marios: could you review https://review.openstack.org/#/c/94810/ please ? | 10:26 |
| lxsli | SpamapS: btw I've abandoned the oac header stuff, there seemed to be no resolution on the spec and the prospect of getting the thing I *actually* wanted (multiple template evaluations) seems so vanishingly small I'm no longer interested in pursuing it | 10:26 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Extract mysql reset files from reset-db https://review.openstack.org/93104 | 10:27 |
| *** lokesh184 has quit IRC | 10:28 | |
| *** markmc has joined #tripleo | 10:29 | |
| SpamapS | lxsli: I understand. The review did turn into a ball of WTF unfortunately. | 10:29 |
| marios | ccorrigan: done | 10:33 |
| *** e0ne has quit IRC | 10:34 | |
| *** e0ne has joined #tripleo | 10:34 | |
| openstackgerrit | A change was merged to openstack/os-apply-config: Fix for string key traversal bug #1321827 https://review.openstack.org/94810 | 10:35 |
| uvirtbot | Launchpad bug 1321827 in os-apply-config "os-apply-config fails with TypeError: string indices must be integers when traversing strings " [Medium,In progress] https://launchpad.net/bugs/1321827 | 10:35 |
| *** e0ne has quit IRC | 10:39 | |
| ccorrigan | marios: thanks | 10:40 |
| *** matsuhashi has quit IRC | 10:41 | |
| *** nosnos has quit IRC | 10:41 | |
| *** nosnos has joined #tripleo | 10:45 | |
| *** lokesh184 has joined #tripleo | 10:46 | |
| *** nosnos has quit IRC | 10:46 | |
| *** shakamunyi has joined #tripleo | 10:47 | |
| *** nosnos has joined #tripleo | 10:47 | |
| *** jml has quit IRC | 10:48 | |
| *** nosnos has quit IRC | 10:48 | |
| *** jml has joined #tripleo | 10:55 | |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack/tripleo-image-elements: Switch from upstart to os-refresh-config for MySQL server-id's https://review.openstack.org/93039 | 10:55 |
| *** shakamunyi has quit IRC | 11:00 | |
| *** rlandy has joined #tripleo | 11:00 | |
| *** e0ne has joined #tripleo | 11:02 | |
| *** hewbrocca has joined #tripleo | 11:04 | |
| *** jml has quit IRC | 11:09 | |
| *** IvanBerezovskiy has joined #tripleo | 11:12 | |
| *** morazi has quit IRC | 11:17 | |
| giulivo | lifeless, you around? again on https://bugs.launchpad.net/tripleo/+bug/1226310 , devananda is more of the idea this should be fixed on the host which hosts virsh | 11:19 |
| uvirtbot | Launchpad bug 1226310 in tripleo "Nova bm operations fail when LIBVIRT_DEFAULT_URI not set" [Medium,Triaged] | 11:19 |
| *** dguerri is now known as _dguerri | 11:20 | |
| giulivo | lifeless, I think the main point there is that we're not connecting to libvirtd, we are actually using ssh and launching virsh from there | 11:21 |
| giulivo | so this should be a setting done on the host where we ssh... which for tripleo means a setting for the tripleo SSH_USER | 11:21 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-heat-templates: Add passthrough configuration glue. https://review.openstack.org/91810 | 11:22 |
| *** akrivoka has joined #tripleo | 11:23 | |
| *** _dguerri is now known as dguerri | 11:24 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 11:24 |
| giulivo | lifeless, after all, a setting for the local tripleo user is what we use also when creating the VMs so setting it for proper virtual_power_driver functionality looks to me as the more manageable solution ... and will probably allow us to close the bug sooner, any strong opinion for not doing it? | 11:24 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Export seed ip instead of gateway ip. https://review.openstack.org/93553 | 11:26 |
| *** shakamunyi has joined #tripleo | 11:27 | |
| *** dshulyak_ has joined #tripleo | 11:28 | |
| *** dshulyak_ has quit IRC | 11:32 | |
| *** viktors has quit IRC | 11:34 | |
| lxsli | I made a diagram showing how images are built + configured using dib, occ, orc + oac. How might I share that upstream? | 11:36 |
| lxsli | please :) | 11:37 |
| *** lokesh184 has quit IRC | 11:38 | |
| TheJulia | lxsli: not that I have an answer for you, but I'd love to see your diagram | 11:38 |
| SpamapS | lxsli: svg works :) | 11:38 |
| lxsli | https://pbs.twimg.com/media/BozOdcDCcAAtI_e.png:large | 11:39 |
| lxsli | SpamapS: where would I put it? tripleo-incubator/docs ? | 11:40 |
| SpamapS | lxsli: if you can save that from the source tool as an svg, you can just land it in incubator or t-i-e's docs dir might work | 11:40 |
| lxsli | urrrgh Sphrinx | 11:40 |
| *** jang has joined #tripleo | 11:41 | |
| lxsli | ok patch inc, thanks | 11:41 |
| *** shakamunyi has quit IRC | 11:41 | |
| rdopiera | if you can't save as svg, save it as pdf and open in inkscape, then save as svg :) | 11:41 |
| lxsli | draw.io ftw, SVG is no problem thanks :) | 11:41 |
| rdopiera | (also if you ever need to edit a pdf, inkscape does it pretty well) | 11:42 |
| SpamapS | yeah I lurve me some inkscape | 11:46 |
| *** dguerri is now known as _dguerri | 11:47 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-incubator: Add an overview diagram to the userguide https://review.openstack.org/96428 | 11:48 |
| *** bcrochet|g0ne is now known as bcrochet | 11:54 | |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Start openvswitch-switch early in upstart https://review.openstack.org/96221 | 11:59 |
| jp_at_hp | SpamapS: you might be interested in ^^^ | 12:05 |
| *** weshay has joined #tripleo | 12:05 | |
| openstackgerrit | Peter Belanyi proposed a change to openstack/tuskar-ui: Add jshint for javascript linting https://review.openstack.org/94866 | 12:05 |
| *** shakamunyi has joined #tripleo | 12:07 | |
| *** jang has quit IRC | 12:08 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Use seed ip instead of gateway ip for ssh-keygen. https://review.openstack.org/94219 | 12:18 |
| *** shakamunyi has quit IRC | 12:20 | |
| *** edmund1 has joined #tripleo | 12:21 | |
| *** _dguerri is now known as dguerri | 12:23 | |
| *** dprince has joined #tripleo | 12:23 | |
| *** rakesh_hs has quit IRC | 12:25 | |
| *** jml has joined #tripleo | 12:27 | |
| *** weshay has quit IRC | 12:27 | |
| *** weshay has joined #tripleo | 12:27 | |
| *** hashar has joined #tripleo | 12:29 | |
| dprince | slagle: would you be up for reviewing this: https://review.openstack.org/#/c/87971/. Would like to get it in for the new batch of TE workers... | 12:30 |
| *** morazi has joined #tripleo | 12:37 | |
| slagle | dprince: sure | 12:38 |
| Ng | morning | 12:39 |
| *** jml_ has joined #tripleo | 12:41 | |
| *** jtomasek has quit IRC | 12:42 | |
| lxsli | TheJulia: didn't mean to ignore you sorry - review is up now: https://review.openstack.org/#/c/96428/ | 12:44 |
| lxsli | http://docs-draft.openstack.org/28/96428/1/check/gate-tripleo-incubator-docs/5aae3e1/doc/build/html/userguide.html | 12:44 |
| lxsli | slagle: Morning, mind adding a comment to your -1 please? https://review.openstack.org/#/c/92400/ | 12:46 |
| *** tzumainn has joined #tripleo | 12:47 | |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Add the ability to create databases from HEAT metadata https://review.openstack.org/96448 | 12:51 |
| *** yamahata has quit IRC | 12:51 | |
| *** noslzzp has joined #tripleo | 12:52 | |
| *** jtomasek has joined #tripleo | 12:55 | |
| slagle | lxsli: it's there now... | 12:55 |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Add the ability to create databases from HEAT metadata https://review.openstack.org/96448 | 12:55 |
| lxsli | slagle: thanks :) | 12:59 |
| *** e0ne_ has joined #tripleo | 13:08 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Add jshint for javascript linting https://review.openstack.org/94866 | 13:10 |
| *** e0ne has quit IRC | 13:10 | |
| *** jdob has joined #tripleo | 13:11 | |
| *** ddieterly has joined #tripleo | 13:13 | |
| *** lazy_prince has quit IRC | 13:14 | |
| *** shakamunyi has joined #tripleo | 13:16 | |
| *** shakayumi has joined #tripleo | 13:18 | |
| *** noslzzp has quit IRC | 13:20 | |
| *** julim has joined #tripleo | 13:21 | |
| *** shakamunyi has quit IRC | 13:21 | |
| *** eghobo has joined #tripleo | 13:22 | |
| *** ramishra has joined #tripleo | 13:29 | |
| *** noslzzp has joined #tripleo | 13:30 | |
| *** casanch1 has joined #tripleo | 13:33 | |
| *** shakayumi has quit IRC | 13:34 | |
| NobodyCam | good morning TripleO | 13:35 |
| NobodyCam | I have a question about OpenSuSe, looking at http://download.opensuse.org/repositories/Cloud:/Images/images/ I see openSUSE-13.1-OS-guest and openSUSE-13.1-OS-Heat-guest images just woundering if we need the Heat version or not? anyon have thoughts or input?? | 13:37 |
| *** yamahata has joined #tripleo | 13:44 | |
| *** morazi has quit IRC | 13:44 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/os-apply-config: Updated from global requirements https://review.openstack.org/96233 | 13:46 |
| *** weshay has quit IRC | 13:46 | |
| *** shakayumi has joined #tripleo | 13:47 | |
| *** matty_dubs|gone is now known as matty_dubs | 13:48 | |
| NobodyCam | undercloud has / uses HEAT_ENV var, but I do not see it in being set in devtest_variables.sh so I'm not sure if its valid to use when building seed | 13:49 |
| *** morazi has joined #tripleo | 13:50 | |
| openstackgerrit | Stuart McLaren proposed a change to openstack/tripleo-incubator: Run the overcloud with an SSL enabled public IP https://review.openstack.org/85098 | 13:53 |
| *** dguerri is now known as _dguerri | 13:58 | |
| *** _dguerri is now known as dguerri | 13:58 | |
| *** weshay has joined #tripleo | 13:58 | |
| openstackgerrit | Loganathan Parthipan proposed a change to openstack/tripleo-incubator: Make disk bus type configurable https://review.openstack.org/92400 | 14:02 |
| lxsli | NobodyCam: iirc HEAT_ENV is set according to script option --heat-env, defaulting to {under,over}cloud-env.json | 14:04 |
| lxsli | Ooh, when building seed... | 14:04 |
| NobodyCam | ya | 14:04 |
| NobodyCam | :-p | 14:05 |
| NobodyCam | OpenSuSe has both options | 14:05 |
| lxsli | The "undercloud Heat" lives in the seed but the seed is bootstrapped not brought up with Heat | 14:07 |
| lxsli | I'm trying to help with your 2nd q btw, not sure how it relates to the first q | 14:07 |
| *** eghobo has quit IRC | 14:08 | |
| NobodyCam | lxsli: I'm reworking the opensuse element and looking at this line: https://github.com/openstack/diskimage-builder/blob/master/elements/opensuse/root.d/10-opensuse-cloud-image#L20 | 14:09 |
| lxsli | NobodyCam: OK, don't think I can help you, sorry | 14:09 |
| NobodyCam | I'm going to go with the non heat version for now. If it crops up in the review process I'll add it in. | 14:10 |
| NobodyCam | lxsli: Thank you it was a help :) | 14:10 |
| *** geerdest has joined #tripleo | 14:12 | |
| lxsli | NobodyCam: seems fair, afaik a single base image is used for all nodes on Ubuntu | 14:12 |
| NobodyCam | :) | 14:14 |
| openstackgerrit | Gerry Drudy proposed a change to openstack/tripleo-image-elements: Configure passthrough in swift config files https://review.openstack.org/92628 | 14:21 |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-image-elements: Change horizon binding address to local-ipv4 in haproxy case https://review.openstack.org/91089 | 14:23 |
| *** eghobo has joined #tripleo | 14:23 | |
| *** eghobo has quit IRC | 14:24 | |
| *** jdonalds has quit IRC | 14:26 | |
| *** ramishra has quit IRC | 14:27 | |
| *** jprovazn has quit IRC | 14:28 | |
| lparth | Could someone please review https://review.openstack.org/#/c/92400/ | 14:31 |
| *** vinsh has joined #tripleo | 14:37 | |
| *** nati_ueno has joined #tripleo | 14:37 | |
| *** weshay has quit IRC | 14:45 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Update bootstack to use sysctl-set-value. https://review.openstack.org/84600 | 14:50 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Use sysctl -p instead of --load https://review.openstack.org/96483 | 14:50 |
| *** rdopiera has quit IRC | 14:51 | |
| *** weshay has joined #tripleo | 14:58 | |
| ccorrigan | derekh_: blocking issue now resolved! Could you re-review https://review.openstack.org/#/c/88340 if you get a chance ? | 14:58 |
| derekh_ | ccorrigan: will do | 14:58 |
| Shrews | Ng: Re: Bug #1324286... If Ironic were to update the instance to store the device labels, so that block-device-mapping would report the labels (I think), would that cause headaches for you guys? | 14:59 |
| uvirtbot | Launchpad bug 1324286 in tripleo "ephemeral partition not being mounted" [High,Triaged] https://launchpad.net/bugs/1324286 | 14:59 |
| Shrews | So instead of "root": "/dev/sda", you'd get "root": "/some-label" | 15:00 |
| Shrews | and same for ephemeral | 15:00 |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1323167 | 15:00 |
| uvirtbot | Launchpad bug 1323167 in tripleo "overcloud can't create an instance by neutron error" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1314978 | 15:00 |
| uvirtbot | Launchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1315355 | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| uvirtbot | Launchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New] | 15:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 15:00 |
| Ng | Shrews: hmm, interesting question. that would come down to what cloud-init would do with those. is that what nova does? | 15:00 |
| ccorrigan | derekh_: thanks | 15:01 |
| derekh_ | ccorrigan: np | 15:01 |
| Shrews | Ng: i have no idea. right now, we aren't setting those values for the instance (because we don't know the actual device names). but we can know the labels | 15:01 |
| Ng | Shrews: I don't think it can make anything worse, because the root fs is already up and running by the time cloud-init parses those mappings ;) | 15:02 |
| Shrews | I should probably learn a bit about cloud-init | 15:03 |
| Ng | Shrews: a very quick scan of the cloud-init code suggests to me that it's not going to grok labels, it's expecting /dev/ items. How was nova-bm able to make this work? | 15:05 |
| Ng | I suppose we might be able to send /dev/disk/by-label/blah | 15:05 |
| Shrews | Ng: nova-bm just set a hard-code value. dunno exactly how that worked: https://github.com/openstack/nova/blob/master/nova/virt/baremetal/driver.py#L225 | 15:06 |
| Ng | Shrews: so, I'd say that the minimal thing to do would be to set that default, since it's pretty reasonable, but we should experiment with something that sets labels (probably including disk/by-label/) to see how it works | 15:08 |
| Ng | I'm slightly concerned about doing anything with the filesystem, because who knows what people will put on their partitions :) | 15:09 |
| Shrews | Ng: I rather like the /dev/disk/by-label solution | 15:09 |
| Ng | Shrews: assuming it works I think it would be sufficient for now, but anyone who has a filesystem that blkid can't grok, is going to be sad | 15:10 |
| Shrews | lucasagomes: ^^^^ | 15:10 |
| Ng | but if we can at least get the default case unbroken for now, that would be a very good start imo :) | 15:10 |
| Shrews | Ng: lucasagomes is the author of most of the partitioning code. he can probably guide us best | 15:11 |
| lucasagomes | Shrews, I also like more the label flexibility, idk cloud-init as well but the /dev/disk/by-label sounds good, can we do a quick testing on it? just add instance.default_ephemeral_device=/dev/disk/by-label/ephemeral0 at the spawn() method in the driver and see if it mounted after the boot | 15:14 |
| Shrews | Ng: if i put up a WIP review with that ^^^, can you test it? | 15:15 |
| *** weshay has quit IRC | 15:16 | |
| *** weshay has joined #tripleo | 15:22 | |
| Ng | Shrews: yep | 15:22 |
| Shrews | cool. gimme a few minues | 15:23 |
| Shrews | minutes, even | 15:23 |
| *** jml_ has quit IRC | 15:26 | |
| *** nati_ueno has quit IRC | 15:27 | |
| *** andreaf has joined #tripleo | 15:28 | |
| *** andreaf has quit IRC | 15:29 | |
| *** blamar has quit IRC | 15:29 | |
| *** BadCub has joined #tripleo | 15:32 | |
| *** BadCub has quit IRC | 15:32 | |
| *** akuznetsov has quit IRC | 15:33 | |
| *** BadCub has joined #tripleo | 15:35 | |
| derekh_ | lifeless: R1 has got no TE hosts running, spent the last few hours trying to bring new ones up but to luck, I think its time to pull the trigger on removing an overcloud job (actually move it to experimental) and temperarily take R1 out of the config while we redeploy/verify | 15:37 |
| Shrews | Ng: https://review.openstack.org/96498 | 15:37 |
| *** jang has joined #tripleo | 15:38 | |
| *** ramishra has joined #tripleo | 15:38 | |
| derekh_ | lifeless: got got patches ready to do above, main question is which overcloud job gets moved ? I think fairest option is to randomize the decision and which ever distro gets overcloud job the other gets undercloud, how would this sound ? | 15:39 |
| anteaya | hello | 15:39 |
| anteaya | so correct me if I am wrong, the tuskar project is now under the tripleo umbrella, yeah? | 15:40 |
| Ng | anteaya: yup | 15:42 |
| anteaya | great | 15:43 |
| *** ramishra has quit IRC | 15:43 | |
| anteaya | so help me understand why tuskar still has its own irc channel? | 15:43 |
| anteaya | https://review.openstack.org/#/c/96160/ | 15:43 |
| anteaya | shouldn't that channel be folded into #tripleo? | 15:44 |
| Ng | Shrews: thanks | 15:44 |
| vinsh | heat is part of tripleo.. but has a very active channel. | 15:44 |
| Ng | heat isn't part of tripleo in the openstack project sense | 15:44 |
| Shrews | Ng: np. that, of course, requires udev, but most systems should have that i think | 15:44 |
| Ng | Shrews: yeah | 15:45 |
| vinsh | ah. | 15:45 |
| Ng | anteaya: it could well be justified that tripleo is a programme, with various things under it | 15:45 |
| anteaya | vinsh: heat is its own program: http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n93 | 15:45 |
| anteaya | Ng it is in my eyes, yes | 15:46 |
| Ng | anteaya: I mean in the sense that, not all of tripleo should necessarily sit in the same channel, but I have zero opinion on whether #tuskar should remain separate | 15:46 |
| anteaya | when reviewing this patch for the #tuskar channel my response is to -1 it and state that tuskar business should be in the tripleo channel | 15:47 |
| *** weshay has quit IRC | 15:47 | |
| Ng | ok now I'm starting to have an opinion ;) | 15:47 |
| anteaya | from an infra point of view, we want folks in places we can find them, not fragmented | 15:47 |
| anteaya | Ng thanks | 15:47 |
| anteaya | I'm interested to hear it | 15:47 |
| *** weshay has joined #tripleo | 15:48 | |
| anteaya | Ng let me know when you are able to articulate your opinion, I need to know how to review that patch | 15:49 |
| *** jcoufal has quit IRC | 15:50 | |
| Ng | anteaya: I'm thinking that the idea of merging the channels would be a topic for later discussion. right now this arrangement seems to be working for all involved (at least I haven't seen any rancour about it), so I'd encourage infra to adopt the status quo and then raise the question separately, if it is causing issues | 15:50 |
| Ng | (because I think that having the tuskar folk get access to the bot's unlimited powers, is more immediately useful than pausing to think about the pros and cons or merging channels, which probably very few of us have done) | 15:50 |
| anteaya | okay fair enough | 15:51 |
| lxsli | Ng: looking for more input on this spec please: https://review.openstack.org/#/c/94910/ | 15:52 |
| anteaya | since I am only +/-1 on this, I will do so with the technical merits of the patch in mind | 15:52 |
| anteaya | Ng I would encourage a discussion so that when/if a +/-2 infra happens upon the patch, there is a log of discussion to point to | 15:53 |
| anteaya | as a heads up, Jim will probably care | 15:53 |
| anteaya | but I'm just guessing | 15:53 |
| openstackgerrit | Martin Geisler proposed a change to openstack/tuskar: Remove © and remove unnecessary encoding lines https://review.openstack.org/96123 | 15:58 |
| *** shakayumi is now known as shakamunyi | 15:59 | |
| Ng | anteaya: would you be happy to start a list thread? | 16:03 |
| *** shakamunyi has left #tripleo | 16:04 | |
| anteaya | I would do that | 16:05 |
| *** shakamunyi has joined #tripleo | 16:05 | |
| anteaya | I don't think i can handle much more anger directed at me this week | 16:05 |
| anteaya | I'm all done for this week | 16:05 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/os-collect-config: Updated from global requirements https://review.openstack.org/96234 | 16:05 |
| anteaya | do you think this is likely to upset people? | 16:05 |
| anteaya | I can do it Monday when I have more wearwithall | 16:05 |
| *** dguerri is now known as _dguerri | 16:07 | |
| *** _dguerri is now known as dguerri | 16:10 | |
| *** nati_ueno has joined #tripleo | 16:12 | |
| Ng | anteaya: I would be disappointed if it elicited angry responses | 16:12 |
| anteaya | okay I will kick it off | 16:13 |
| anteaya | mostly like, this has come up, how do people feel about it | 16:13 |
| anteaya | how's that for an approach? | 16:13 |
| *** martyntaylor has quit IRC | 16:14 | |
| Ng | anteaya: sounds fine to me | 16:14 |
| *** matty_dubs is now known as matty_dubs|lunch | 16:14 | |
| anteaya | k, thanks | 16:15 |
| mestery | tchaypo: The fix for "OVS restart" was merged today for this bug: https://bugs.launchpad.net/neutron/+bug/1290486 | 16:16 |
| uvirtbot | Launchpad bug 1290486 in tripleo "neutron-openvswitch-agent does not recreate flows after ovsdb-server restarts" [Critical,In progress] | 16:16 |
| mestery | tchaypo: Let me know if you still see problems with OVS restarts on tripleo. | 16:16 |
| *** blamar has joined #tripleo | 16:16 | |
| *** morazi has quit IRC | 16:18 | |
| *** IvanBerezovskiy has left #tripleo | 16:20 | |
| lxsli | getting tired of "recheck no bug" on https://review.openstack.org/#/c/94856/ , it has passed all four checks at some point, any chance it could get pushed through? | 16:20 |
| *** hashar has quit IRC | 16:22 | |
| TheJulia | win 6 | 16:24 |
| anteaya | Ng: http://lists.openstack.org/pipermail/openstack-dev/2014-May/036265.html | 16:26 |
| *** nati_ueno has quit IRC | 16:26 | |
| *** giulivo has quit IRC | 16:27 | |
| *** e0ne_ has quit IRC | 16:30 | |
| *** morazi has joined #tripleo | 16:30 | |
| *** e0ne has joined #tripleo | 16:31 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Wait for ssh demon to become active https://review.openstack.org/96515 | 16:31 |
| *** martyntaylor has joined #tripleo | 16:33 | |
| *** e0ne has quit IRC | 16:35 | |
| Ng | anteaya: groovy | 16:40 |
| openstackgerrit | Stuart McLaren proposed a change to openstack/tripleo-incubator: Run the overcloud with an SSL enabled public IP https://review.openstack.org/85098 | 16:43 |
| *** TravT has joined #tripleo | 16:50 | |
| *** jistr is now known as jistr|afk | 16:56 | |
| *** lucasagomes has quit IRC | 16:59 | |
| *** shardy is now known as shardy_afk | 17:01 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Export seed ip instead of gateway ip. https://review.openstack.org/93553 | 17:06 |
| *** matty_dubs|lunch is now known as matty_dubs | 17:14 | |
| *** jistr|afk has quit IRC | 17:16 | |
| *** derekh_ has quit IRC | 17:18 | |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-image-elements: Add a default signing directory for ceilometer-api to avoid the performance issues created by keystone client use of temp directory (default behavior when not specified). https://review.openstack.org/96527 | 17:20 |
| *** markmc has quit IRC | 17:21 | |
| *** csd has joined #tripleo | 17:26 | |
| *** dshulyak_ has joined #tripleo | 17:33 | |
| *** eghobo has joined #tripleo | 17:34 | |
| *** nati_ueno has joined #tripleo | 17:37 | |
| *** rpodolyaka1 has joined #tripleo | 17:43 | |
| *** akuznetsov has joined #tripleo | 17:47 | |
| *** pblaho has quit IRC | 17:47 | |
| *** rpodolyaka1 has quit IRC | 17:55 | |
| *** martyntaylor has left #tripleo | 17:56 | |
| *** openstackgerrit has quit IRC | 17:57 | |
| *** yamahata has quit IRC | 17:59 | |
| *** edmund1 has quit IRC | 18:01 | |
| *** edmund has joined #tripleo | 18:01 | |
| *** rpodolyaka1 has joined #tripleo | 18:05 | |
| *** edmund has quit IRC | 18:10 | |
| *** akrivoka has quit IRC | 18:11 | |
| *** vinsh has quit IRC | 18:12 | |
| *** dkehn has quit IRC | 18:15 | |
| *** dkehn has joined #tripleo | 18:17 | |
| *** pelix has quit IRC | 18:21 | |
| *** hashar has joined #tripleo | 18:28 | |
| *** rpodolyaka1 has quit IRC | 18:28 | |
| *** rpodolyaka1 has joined #tripleo | 18:32 | |
| *** morganfainberg is now known as morganfainberg_Z | 18:33 | |
| *** morganfainberg_Z is now known as morganfainberg | 18:37 | |
| *** akuznetsov has quit IRC | 18:38 | |
| *** hashar has quit IRC | 18:41 | |
| *** akuznetsov has joined #tripleo | 18:42 | |
| *** csd has quit IRC | 18:43 | |
| *** jp_at_hp has quit IRC | 18:45 | |
| *** rpodolyaka1 has quit IRC | 18:47 | |
| *** hashar has joined #tripleo | 18:49 | |
| *** geerdest has quit IRC | 18:52 | |
| *** akuznetsov has quit IRC | 19:00 | |
| lifeless | greghaynes: btw you should live in #openstack-dev too | 19:01 |
| greghaynes | oh, how did I not knpw of this | 19:03 |
| *** e0ne has joined #tripleo | 19:07 | |
| *** rpodolyaka1 has joined #tripleo | 19:07 | |
| *** openstackgerrit has joined #tripleo | 19:07 | |
| *** jang1 has joined #tripleo | 19:07 | |
| *** jang has quit IRC | 19:11 | |
| *** akuznetsov has joined #tripleo | 19:13 | |
| *** openstackgerrit has quit IRC | 19:13 | |
| *** nati_ueno has quit IRC | 19:16 | |
| *** dprince has quit IRC | 19:19 | |
| *** hashar has quit IRC | 19:21 | |
| *** akuznetsov has quit IRC | 19:21 | |
| *** eghobo has quit IRC | 19:23 | |
| lifeless | adam_g: Ng: so the list of patches being apply | 19:23 |
| *** eghobo has joined #tripleo | 19:23 | |
| lifeless | adam_g: we had multiple copies when I went to sleep; whats the current status of consolidating that ? | 19:23 |
| Ng | https://etherpad.openstack.org/p/tripleo-end-to-end-patches is the list wrt 405 tests | 19:25 |
| *** derekh_ has joined #tripleo | 19:27 | |
| adam_g | lifeless, i threw that stuff in an etherpad as a pretty summary, but we're using this to actually manage what we're pulling in, so its probably more accurate https://github.com/gandelman-a/tripleo-end-to-end/blob/master/repo_refs.yaml | 19:28 |
| *** rpodolyaka1 has quit IRC | 19:28 | |
| derekh_ | lifeless: on train at the moment, just wanted to make sure you saw them, the patches I mentioned earlier to remove R1 -- https://review.openstack.org/#/c/96546/ https://review.openstack.org/#/c/96547/ | 19:28 |
| *** nati_ueno has joined #tripleo | 19:28 | |
| *** jrist has quit IRC | 19:30 | |
| lifeless | adam_g: I'd like there to be Just One Copy if possible | 19:31 |
| lifeless | derekh_: cool; though I'm sad at the choice of Ubuntu there ;> | 19:32 |
| lifeless | derekh_: isn't ironic-undercloud already non-voting | 19:32 |
| lifeless | derekh_: I've gotten hold of fw update tools for our hardware | 19:33 |
| lifeless | going to apply those today | 19:33 |
| *** rlandy has quit IRC | 19:33 | |
| derekh_ | lifeless: ya it is non voting, but I'm triing to reduce demand so R2 can keep up | 19:33 |
| lifeless | derekh_: I thought it was experimental, I meant. | 19:34 |
| lifeless | but clearly am wrong | 19:34 |
| derekh_ | lifeless: hopefully we'll be it all back ASAP, I can pretty much spend as most my time getting us back running | 19:34 |
| derekh_ | lifeless: sorry about the choice, shouldn't be too long | 19:35 |
| derekh_ | lifeless: ya it was experimental for most jobs | 19:35 |
| derekh_ | lifeless: but this moves it for the others | 19:35 |
| *** openstackgerrit has joined #tripleo | 19:35 | |
| *** nati_ueno has quit IRC | 19:36 | |
| derekh_ | lifeless: btw have you seen this before, I tried several times to kick off new TE hosts on R1 today | 19:36 |
| *** nati_ueno has joined #tripleo | 19:36 | |
| derekh_ | lifeless: each time they timedout | 19:36 |
| lifeless | derekh_: I have +1'd - go advocate in infra :) | 19:36 |
| derekh_ | lifeless: textcons just told me | 19:36 |
| derekh_ | lifeless: "Monitor is in graphics mode or an unsupported text mode." | 19:36 |
| derekh_ | lifeless: will do | 19:37 |
| lifeless | Its possible that we have an altered kernel config disabling the trusty text-mode thing | 19:37 |
| lifeless | or | 19:37 |
| lifeless | no, fairly sure its that | 19:37 |
| lifeless | altered or old | 19:37 |
| * Ng sighs | 19:39 | |
| Ng | that problem is never going to fully go away, is it | 19:39 |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-image-elements: Add default signing directory for keystone client https://review.openstack.org/96527 | 19:41 |
| derekh_ | lifeless: so this was the same test-env image we were previously using, actually I'm not sure if that was trusty or saucy | 19:41 |
| lifeless | derekh_: should be trusty by now :) | 19:42 |
| *** jrist has joined #tripleo | 19:44 | |
| *** ifarkas has quit IRC | 19:49 | |
| *** dshulyak_ has quit IRC | 19:53 | |
| *** nati_ueno has quit IRC | 19:54 | |
| *** markmc has joined #tripleo | 19:55 | |
| *** markmc has quit IRC | 19:55 | |
| derekh_ | lifeless: so if your do firmware update, whats next burn in test ? we should probably do undercloud and seed also i.e. start at the start | 19:56 |
| derekh_ | lifeless: have ye (NOC ? ) and standard burn in test ? or should I do some research to see what that standard is ? | 19:57 |
| *** nati_ueno has joined #tripleo | 19:57 | |
| lifeless | derekh_: so I rather suspect our ideal burn-in test is to run the cloud | 19:58 |
| lifeless | derekh_: doing jobs | 19:59 |
| lifeless | derekh_: like make a period job that runs 500 times a day or something | 19:59 |
| lifeless | derekh_: or bring up our own nodepool etc - but I'd like to avoid that if possible | 19:59 |
| lifeless | ok so lesson one on specs | 20:00 |
| lifeless | we need to get detailed ;) | 20:00 |
| derekh_ | lifeless: ok sounds fine by me, we can keep in away from infra until we have lots of testing done, I'll add more detail to spec | 20:01 |
| lifeless | derekh_: so I meant run an infra periodic job | 20:01 |
| derekh_ | lifeless: we can do a crap load of tests without nodepool / geaman / zuul | 20:01 |
| lifeless | lets nab jeblair | 20:01 |
| *** nati_ueno has quit IRC | 20:01 | |
| derekh_ | lifeless: I'm thinking we can keep it out of infra, at least until we verify the baseline is no worse then R2 | 20:02 |
| lifeless | derekh_: maybe - lets have a discussion with infra | 20:03 |
| openstackgerrit | Jay Dobies proposed a change to openstack/tripleo-specs: Spec for Tuskar Juno REST API changes https://review.openstack.org/94720 | 20:03 |
| derekh_ | lifeless: ok cool | 20:03 |
| *** nati_ueno has joined #tripleo | 20:03 | |
| derekh_ | lifeless: if I drop off, train probably entered a black hole | 20:04 |
| lifeless | so if you drop off the earth is going to die? :) | 20:04 |
| TheJulia | well, it could just be a micro black hole | 20:05 |
| derekh_ | lifeless: afraid so, don't worry you wont even know it happened | 20:05 |
| lifeless | :) | 20:05 |
| *** jeblair has joined #tripleo | 20:06 | |
| derekh_ | lifeless: so basically what I'm trying to avoid is us getting deployed and moving straight to infra only to find a obvious problem we could have spotted ourselves | 20:07 |
| lifeless | so that doesn't worry me so much | 20:07 |
| lifeless | what worries me is | 20:08 |
| lifeless | - external network glitchiness | 20:08 |
| lifeless | - the 33% failure rate stuff we have happening from time to time | 20:08 |
| lifeless | - reliability talking to e.g. testenvs | 20:08 |
| derekh_ | lifeless: your periodic job is good but we shouldn't go straight to it, | 20:08 |
| lifeless | - the 'up for 3 days' stuff we've had with mellanox | 20:08 |
| derekh_ | lifeless: yes and we can debug those without truning things on in infra | 20:09 |
| derekh_ | *turning | 20:09 |
| lifeless | you clearly have something in mind :) | 20:09 |
| derekh_ | lifeless: pretty much what your have suggested execpt we drive it outside of infra for a few days | 20:10 |
| lifeless | derekh_: I'd like something thats turnkey | 20:10 |
| lifeless | derekh_: so we're not using a few days of a tripleo-cd-admin time to do this each time we have to do it | 20:11 |
| derekh_ | lifeless: I'll set up a jenkins (or many), if everything looks ok after a few days , then config the region back to infra | 20:11 |
| derekh_ | lifeless: ok, we can create something driven by upstream config that can be reused | 20:12 |
| *** saurabhs has joined #tripleo | 20:12 | |
| derekh_ | lifeless: ok that bit I have thought fully through but we should be able to do something reusable | 20:13 |
| derekh_ | lifeless: the baseline acceptance level to accept into production would be "failure rate no worse then current regions" | 20:13 |
| lifeless | sure | 20:14 |
| *** nati_ueno has quit IRC | 20:15 | |
| derekh_ | lifeless: opps typo "ok that bit I have NOT thought fully through but we should be able to do something reusable" | 20:16 |
| derekh_ | so the complete opposite to what I said | 20:16 |
| *** jtomasek has quit IRC | 20:18 | |
| *** derekh_ has quit IRC | 20:19 | |
| -openstackstatus- NOTICE: Gerrit is going offline to correct an issue with a recent project rename. ETA 20:45 UTC. | 20:22 | |
| *** ChanServ changes topic to "Gerrit is going offline to correct an issue with a recent project rename. ETA 20:45 UTC." | 20:22 | |
| jerryz | lifeless: i just got nodepool sshclient to work with paramiko proxy. which do you think it's better to get the config from, parsing a ssh_config file per cloud provider or add a proxy command field in nodepool.yaml | 20:23 |
| *** edmund has joined #tripleo | 20:25 | |
| *** openstackgerrit has quit IRC | 20:25 | |
| lifeless | jerryz: I don't know. keeping all the config in nodepool.yaml has some appeal | 20:25 |
| * clarkb will mention the nodepool discussion happens over in -infra >_> | 20:33 | |
| clarkb | and as soon as gerrit is happy again I am sure we will be happy to discuss it | 20:33 |
| lifeless | clarkb: indeed :) | 20:35 |
| *** vinsh has joined #tripleo | 20:36 | |
| *** ChanServ changes topic to "https://etherpad.openstack.org/p/tripleo-ci-r1-trusty | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900//0700 UTC in #openstack-meeting-alt" | 20:40 | |
| -openstackstatus- NOTICE: Gerrit is back online | 20:41 | |
| *** chuckC has quit IRC | 20:44 | |
| *** edmund has quit IRC | 20:51 | |
| *** edmund1 has joined #tripleo | 20:51 | |
| *** shakamunyi has quit IRC | 20:51 | |
| *** shakamunyi has joined #tripleo | 20:52 | |
| *** olaph has joined #tripleo | 20:53 | |
| *** jdob has quit IRC | 20:54 | |
| *** rcarrill` has joined #tripleo | 20:58 | |
| tchaypo | mestery: will do | 20:59 |
| mestery | tchaypo: There's also this: https://review.openstack.org/#/c/96555/ | 20:59 |
| mestery | tchaypo: Found by myself and someone from RAX during testing. | 21:00 |
| mestery | tchaypo: Hopefully that merges today as well. | 21:00 |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1323167 | 21:00 |
| uvirtbot | Launchpad bug 1323167 in tripleo "overcloud can't create an instance by neutron error" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1314978 | 21:00 |
| uvirtbot | Launchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1324598 | 21:00 |
| uvirtbot | Launchpad bug 1324598 in tripleo "Passthrough-enable for nova.conf mustache template is not correct" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1315355 | 21:00 |
| uvirtbot | Launchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 21:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| *** rcarrillocruz has quit IRC | 21:00 | |
| *** geerdest has joined #tripleo | 21:02 | |
| tchaypo | Haha | 21:02 |
| *** matty_dubs is now known as matty_dubs|gone | 21:04 | |
| mestery | tchaypo: Is that Haha regarding bug 1324598? :) | 21:09 |
| uvirtbot | Launchpad bug 1324598 in tripleo "Passthrough-enable for nova.conf mustache template is not correct" [High,Triaged] https://launchpad.net/bugs/1324598 | 21:09 |
| tchaypo | Yes | 21:09 |
| mestery | Heh | 21:10 |
| *** casanch1 has quit IRC | 21:13 | |
| *** shakamunyi has quit IRC | 21:14 | |
| geerdest | what is the problem with this broken voting fedora job? https://review.openstack.org/#/c/94633/1 | 21:14 |
| lifeless | didn't get a test environment | 21:18 |
| lifeless | lets see | 21:18 |
| *** cwolferh has joined #tripleo | 21:24 | |
| *** julim has quit IRC | 21:24 | |
| *** cwolferh_ has quit IRC | 21:24 | |
| *** eguz has joined #tripleo | 21:26 | |
| *** Penick has joined #tripleo | 21:29 | |
| bnemec | I feel like I've seen that failure a bunch today for some reason. | 21:30 |
| lifeless | yes | 21:30 |
| *** morazi has quit IRC | 21:30 | |
| lifeless | either derekh took down the testenvs or they failed | 21:30 |
| lifeless | I haven't looked into the incident log yet | 21:30 |
| lifeless | https://review.openstack.org/#/c/96547/ will fix it once its live | 21:30 |
| *** eghobo has quit IRC | 21:30 | |
| *** nati_ueno has joined #tripleo | 21:31 | |
| *** openstackgerrit has joined #tripleo | 21:35 | |
| *** eghobo has joined #tripleo | 21:38 | |
| greghaynes | Ive also been seeing that all over since last night :( | 21:39 |
| *** eguz_ has joined #tripleo | 21:39 | |
| *** eguz has quit IRC | 21:39 | |
| *** eghobo has quit IRC | 21:43 | |
| *** openstackgerrit has quit IRC | 21:47 | |
| lifeless | ok recheck those things | 21:48 |
| lifeless | they shouldn't hit that again | 21:48 |
| * greghaynes rolls recheck no bug dice | 21:49 | |
| tchaypo | How about adding "recheck no dice" as a synonym? | 21:52 |
| tchaypo | To be used in the times when we're pretty sure it will fail? | 21:52 |
| *** chuckC has joined #tripleo | 21:52 | |
| clarkb | you could just leave a -1 'This change won't pass tests' comment for that | 21:53 |
| tchaypo | Boring | 21:53 |
| tchaypo | Practical | 21:54 |
| tchaypo | But boring | 21:54 |
| *** eguz_ has quit IRC | 22:03 | |
| *** eghobo has joined #tripleo | 22:03 | |
| SpamapS | goood morning from wakcy 4 day jetlag land. | 22:09 |
| *** weshay has quit IRC | 22:10 | |
| *** shakamunyi has joined #tripleo | 22:11 | |
| lifeless | SpamapS: hey cool | 22:13 |
| lifeless | SpamapS: so, up for a brainstorm? also have some product stuff to update you on | 22:14 |
| SpamapS | lifeless: sure.. just having a cup of tea and a cookie. ;) | 22:14 |
| lifeless | hangout? | 22:14 |
| *** shakayumi has joined #tripleo | 22:21 | |
| *** shakamunyi has quit IRC | 22:22 | |
| *** nati_ueno has quit IRC | 22:22 | |
| *** yamahata has joined #tripleo | 22:28 | |
| *** jp_at_hp has joined #tripleo | 22:39 | |
| *** olaph has quit IRC | 22:46 | |
| *** nati_ueno has joined #tripleo | 22:56 | |
| greghaynes | https://review.openstack.org/#/c/96052/ seems to have some interesting fail as a result of some interesting fail | 23:00 |
| *** noslzzp has quit IRC | 23:00 | |
| greghaynes | in the nova-compute log | 23:01 |
| *** vinsh has quit IRC | 23:03 | |
| *** hewbrocca has quit IRC | 23:05 | |
| lifeless | nice... xception during message handling: Circular reference detected | 23:06 |
| lifeless | recheck time | 23:06 |
| *** derekh_ has joined #tripleo | 23:09 | |
| lifeless | derekh_: hey | 23:10 |
| lifeless | HP1 is out of rotation | 23:10 |
| derekh_ | lifeless: hi ya, cooleo | 23:10 |
| derekh_ | lifeless: so anything I can do or just wait for firmware update? can work on burn in stuff while waiting | 23:14 |
| lifeless | work up the burn in stuff I think | 23:16 |
| lifeless | I have to ring AMEX then I'll log into the vpn and download the firmware stuff | 23:16 |
| lifeless | I'm going to prep an entire redeploy of the cloud too I think since its out of rotation | 23:16 |
| lifeless | derekh_: in fact, I think i'm going to nuke the entire thing from ground up | 23:17 |
| derekh_ | lifeless: ok, willdo, when it comes to redeploying the ci overcloud would be good to see if this script is helpfull/suitable https://review.openstack.org/#/c/90146/ | 23:19 |
| *** yamahata has quit IRC | 23:20 | |
| *** ddieterly has quit IRC | 23:27 | |
| *** xuhaiwei has joined #tripleo | 23:38 | |
| *** chuckC has quit IRC | 23:41 | |
| xuhaiwei | morning | 23:41 |
| *** derekh_ has quit IRC | 23:44 | |
| *** ddieterly has joined #tripleo | 23:51 | |
| SpamapS | lifeless: Heat email sent. | 23:52 |
| *** chuckC has joined #tripleo | 23:52 | |
| lifeless | SpamapS: \o/ | 23:53 |
| SpamapS | lifeless: and I thought of another strategy while writing it | 23:57 |
| lifeless | cool | 23:57 |
| SpamapS | lifeless: we could just go into the Heat db and fix things | 23:57 |
| SpamapS | its a horrible strategy long term | 23:57 |
| SpamapS | but short term, it could work | 23:57 |
| lifeless | I shudder to think what 'fix' means in this context | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
