Friday, 2017-04-14

*** john-davidge has joined #openstack-infra00:03
*** ijw has quit IRC00:04
*** jogo has quit IRC00:05
*** jogo has joined #openstack-infra00:06
*** jogo has joined #openstack-infra00:06
*** wolverineav has quit IRC00:07
*** esikache1 has quit IRC00:07
*** john-davidge has quit IRC00:08
*** wolverineav has joined #openstack-infra00:09
*** baoli_ has quit IRC00:10
*** wolverineav has quit IRC00:13
*** scottda has quit IRC00:15
*** kzaitsev_mb has joined #openstack-infra00:15
*** MasterOfBugs has joined #openstack-infra00:16
clarkbrcarrillocruz: cmurphy: hey so I rebooted baremetal00.vanilla.ic.openstack.org to pick up kernel patch whcih seemed like a great idea at the time. But then it never came back with networking. Hopping on the ILO I see the server is up but not doing much. I think it may not be configuring networking on its nic. Do we dhcp for that or is that static or?00:17
clarkbrcarrillocruz: cmurphy: any help debugging that much appreciated as I'm somewhat confused right now00:17
clarkbrcarrillocruz: cmurphy: I did manage to follow a boot with textcons on ilo and it didn't look like the hardware was unhappy or anything00:18
clarkbrcarrillocruz: cmurphy VSP was not working so I couldn't get a login to poke around and see what might be wrong00:19
clarkbanyways, help is much appreciated as I am currently stumped as to what could be unhappy there00:19
clarkbwhat bootstraps the network on the baremetal node?00:20
clarkbis it dhcp?00:20
mnaserclarkb do you have access to another host on that network00:22
mnaseryou could run tcpdump and see if its broadcasting dhcp requests00:22
clarkboh good idea00:22
*** kzaitsev_mb has quit IRC00:23
mnaserclarkb to make life easier, you can use filter 'port 67 or port 68'00:24
mnaserthat'll narrow down dhcp traffic only00:24
clarkbI've got it running however looking at the network config of the host I'm on I doubt its dhcping00:24
clarkbI remember now that its a funny static setup, which confuzzles me more because that should just work :/00:24
mnaserilo doesnt have a kvm console?00:25
* fungi is not really here, but wonders if the switchport is shutdown, maybe loop detection gone awry or something00:25
clarkbmnaser: not that I see, virtual serial which does nothing and the text console which is just dmesg output00:25
mnaserclarkb ilo doesn't have kvm? (we only roll dell gear so im not sure on the feature parity)00:26
fungivirtual serial would probably work if the server was set up to run a getty on that tty00:26
*** hongbin has quit IRC00:26
mnaserif you have access to the switches (and they're cisco)00:27
fungino login prompt if you send newlines through the text console though?00:27
mnasershow interfaces status err-disable will show you if its disabled00:27
clarkbfungi: nope00:27
clarkbmnaser: ya no access unfortunately00:27
fungidiagnosing the switchgear would be easy enough _if_ we had access, yes00:27
mnaserdo they only have 1 port?00:28
*** Julien-zte has joined #openstack-infra00:28
mnaseraaand are they all sitting in the same l2 network?  maybe if routes are broken it is still has l3 access00:28
clarkbI tried the private address locally00:29
*** nemish_m has quit IRC00:29
clarkbguess  I can try the public address too00:29
fungiwould be od for that to change just by rebooting the server itself00:29
clarkbno route to host00:29
clarkbfungi: well we also updated the kernel00:29
clarkbwhich has me worried that maybe the mellanox drivers stopped working00:30
mnaserhave teh server been rebooted before00:30
mnaserhas the* (jeez)00:30
clarkbmnaser: yes I actually rebooted it just before on the old kernel and that was fine00:30
* fungi shakes fist and swears at uselkess "complimentary" wifi here. will let people who are on real network connections try to discuss00:30
clarkbmnaser: then I upgraded the kernel, rebooted and nothing00:30
*** tphummel has quit IRC00:30
clarkbpretty sure most of the boot succeeded based on output from textcons00:30
clarkbit looks like linux dmesg and lots of usb chatter00:31
mnaserwhere does the text console stop at? (or does it clear out at some point)00:31
clarkbits just spamming usb device addition stuff00:31
clarkbfor virtual keyboard from the ilo00:31
mnaseri mean if you have a virtual keyboard could you not break the boot process00:32
mnaserand select the older kernel00:32
clarkbactually probably let me try that00:32
mnasercombo between escape and arrow up/arrow down00:32
mnaser*if* that works but doesnt boot into the kernel, we can always boot into single user mode or change the kernel args to fix the display00:33
clarkbnow we wait for the 10 minute boot process00:35
*** tosky has quit IRC00:35
clarkbthis is why we have VMs00:35
mnaserclarkb the luxury of booting VMs is underrated00:37
mnaser:P00:37
clarkbok its booting old kernel now00:40
clarkbdmesg output looks more "complete"00:40
clarkband it pings00:40
clarkbOK thats really weird, how did I break the kernel update00:40
mnaserclarkb cool, at least we got it back, what was the new kernel version?00:41
clarkbmnaser: latest for trusty 3.13.0-11600:41
clarkbbootable is -7000:41
clarkbwell its not got working ssh yet (boot is waiting for network config to finish)00:42
clarkbbut it pings so I am hopeful00:42
clarkbyup I'm in00:42
mnaserid check system logs00:42
mnaserto see if the previous boots00:42
mnasercompleted or not00:42
clarkbbased on dmesg output I don't think they did00:42
mnaserdmesg in /var/log/dmesg or typing "dmesg"00:42
clarkbthere is far more on the -70 boot so I'm guessing I derped the kernel uinstall00:42
mnaserdmesg will always start from current boot afaik00:43
clarkbmnaser: ya00:43
mnaser /var/log/syslog will give you the more accurate information00:43
clarkbI noticed the server hadn't auto updated the kernel either which may be related00:43
*** EricGonczer_ has joined #openstack-infra00:43
mnaserwell, at least its running now :D00:44
clarkbgrepping /var/log I don't think -116 ever got into proper boot00:44
clarkbfar less content then grepping -7000:44
clarkbrcarrillocruz: cmurphy: so now I will change my question to hwo does one properly upgrade the kernel on these boxes? is it the mellanox hardware that makes it special or?00:45
*** beslemon has joined #openstack-infra00:45
*** nemish_m has joined #openstack-infra00:47
*** baoli has joined #openstack-infra00:48
*** baoli has quit IRC00:48
*** beslemon has quit IRC00:48
fungiahh, wonder if klms need rebuilding against newer headers00:49
fungilkms00:49
*** beslemon has joined #openstack-infra00:49
*** baoli has joined #openstack-infra00:49
clarkbya poking around it may be that we don't have "linux" installed for some reason00:49
*** nemish_m has quit IRC00:50
*** zhenguo has joined #openstack-infra00:52
*** links has joined #openstack-infra00:53
*** inc0 has quit IRC00:54
*** EricGonczer_ has quit IRC00:56
*** linzi has quit IRC01:00
*** linzi has joined #openstack-infra01:00
*** ijw has joined #openstack-infra01:01
mnaseris my internet broken?01:02
mnaserwiki.openstack.org took too long to respond.01:02
mnaserlooks like its taking a long time to respond for both ipv4 locally and from a vm over ipv6 (curl http://wiki.openstack.org/)01:02
*** Apoorva_ has quit IRC01:04
*** Apoorva has joined #openstack-infra01:04
clarkbmtr segfaults on me thats new01:04
clarkbyes it is not responding to ssh even01:04
clarkbfungi: jpmaxman ^ any idea if that is realted to planned work?01:05
clarkbI've got to pop out now for dinner01:07
fungidoesn't sound familiar. checking up on it01:07
fungias much as i can over this connection anyway01:08
fungiwondering if there's a host crash in rackspace01:08
*** ijw has quit IRC01:10
fungicacti seems to have ceased receiving responses from it as of a few minutes ago too01:12
*** kevinshort has quit IRC01:12
*** srobert has joined #openstack-infra01:14
fungihas ping responses though01:14
mnaserfungi if your connection helps at all, checking the console logs might give hints (console log show <uuid>)01:15
mnaserif that doesnt even respond then the host is probably crashed01:15
* mnaser ^01:17
* mnaser &01:17
*** srobert has quit IRC01:19
*** kzaitsev_mb has joined #openstack-infra01:20
fungithe wifi here goes dead for a minute or more at a time and then i get a few seconds of packets through to home... frustrating01:20
*** Julien-zte has quit IRC01:20
*** Julien-zte has joined #openstack-infra01:21
fungi"there is no such action"01:22
*** gyee has quit IRC01:22
fungieven on working instances01:23
fungii don't think rackspace enables it01:23
fungii've always had to use there java-based virtual console but don't have a working java stack on my travel machine01:24
fungis/there/thier/01:25
*** esikache1 has joined #openstack-infra01:25
fungigrr01:26
*** ijw has joined #openstack-infra01:27
fungiinfra-root: anyone around a working computer who can jump into rackspace's portal and check tickets and virtual console for wiki-upgrade-test.openstack.org (the server currently serving the wiki.o.o site)?01:28
*** ijw has quit IRC01:28
*** thorst has quit IRC01:29
*** ijw has joined #openstack-infra01:29
fungithe most i can do from here is maybe hack out a nova reboot command and hope for the best01:29
*** harlowja has quit IRC01:29
fungijhesketh: ^ ?01:29
*** esikache1 has quit IRC01:30
*** Julien-zte has quit IRC01:30
fungiclarkb: pabelanger: what are the odds that the wiki server was in the batch of things you rebooted for new kernels? wondering if maybe something like it came up with no firewall rules and got its elasticsearch hacked again01:33
*** yanyanhu has joined #openstack-infra01:34
jheskethfungi: yep, can take a look01:34
fungiyou're a lifesaver jhesketh. i can't do anything from here and need to get a few hours sleep before heading to an airport01:35
jheskethno worries :-)01:35
*** kzaitsev_mb has quit IRC01:35
jheskeththere are some maintenances scheduled, but they should be complete unless they are running late01:42
openstackgerritMerged openstack-infra/tripleo-ci master: Remove installation of moreutils and replace ts  https://review.openstack.org/45665401:42
*** beslemon_ has joined #openstack-infra01:44
jheskethThe console has a bunch of xenwatch info's about hanging01:45
jheskethA restart might be best01:45
jheskethfungi: want me to reboot it?01:45
fungisure, thanks for checking01:45
*** MasterOfBugs has quit IRC01:45
*** MasterOfBugs has joined #openstack-infra01:46
fungii'll see if i can manage to get in and double-check it's actually firewalled still01:46
jheskethfungi: firewalled where? (can I check to save you)01:46
fungiiptables -L01:47
*** beslemon has quit IRC01:47
jheskethfungi: I can't seem to even ssh in from puppetmaster01:47
jheskethand ctrl-alt-delete on the console seems to have done nothing01:47
jheskethso I think it needs a hard reset01:47
fungiand ip6tables -L01:47
fungionc you're fone rebooting01:47
jheskethokay, I'll do a hard01:47
fungiapologies for typos, can't see half of what i'm typing due to lag (eve _with_ mosh)01:48
jheskethit's doing a fsck01:48
fungiexpected01:49
*** liujiong has joined #openstack-infra01:50
jhesketh30%..01:52
jheskethfungi: booted and appears to be working01:57
jheskethiptables look as expected01:57
fungifinished i guess01:57
fungiyep01:57
fungiiptables output lgtm01:57
*** thorst has joined #openstack-infra01:58
*** thorst has quit IRC01:58
fungiso strange, but glad it wasn't that at least01:58
jheskethyeah, I think something was screwy with the xen host01:59
jheskethlikely following the network maintenence01:59
fungithanks for taking care of it, jhesketh!02:00
jheskethno worries :-)02:00
* fungi disappears02:00
*** ihrachys has joined #openstack-infra02:00
jhesketh#status log wiki.o.o required a hard restart due to host issues following rackspace network maintenence02:01
openstackstatusjhesketh: finished logging02:01
*** esikache1 has joined #openstack-infra02:03
*** esikache1 has quit IRC02:08
*** ijw has quit IRC02:10
*** Apoorva has quit IRC02:12
*** esberglu has joined #openstack-infra02:13
*** esberglu has quit IRC02:13
*** esberglu has joined #openstack-infra02:13
*** esikache1 has joined #openstack-infra02:14
*** esberglu has quit IRC02:17
*** esikache1 has quit IRC02:19
*** rhallisey has quit IRC02:21
*** esikache1 has joined #openstack-infra02:26
*** Apoorva has joined #openstack-infra02:28
*** thorst has joined #openstack-infra02:29
*** esikache1 has quit IRC02:30
*** kzaitsev_mb has joined #openstack-infra02:32
*** zhurong has joined #openstack-infra02:33
*** Apoorva has quit IRC02:34
openstackgerritMatthew Treinish proposed openstack-infra/puppet-subunit2sql master: Config is a dict so use get not getattr  https://review.openstack.org/45681102:34
mtreinishclarkb: ^^^ that's the fix so the subunit worker will start again02:35
*** MasterOfBugs has quit IRC02:36
*** Kevin_Zheng has joined #openstack-infra02:36
*** MasterOfBugs has joined #openstack-infra02:37
*** Julien-zte has joined #openstack-infra02:37
*** kzaitsev_mb has quit IRC02:37
*** ijw has joined #openstack-infra02:39
*** yonglihe has joined #openstack-infra02:39
*** EricGonczer_ has joined #openstack-infra02:43
*** ijw has quit IRC02:44
*** tphummel has joined #openstack-infra02:47
*** thorst has quit IRC02:48
*** gouthamr has quit IRC02:49
*** esikache1 has joined #openstack-infra02:50
*** knangia has quit IRC02:51
*** esikache1 has quit IRC02:55
*** EricGonczer_ has quit IRC02:57
cmurphyclarkb: I'm not really around today, and I don't have an answer about kernel updates on those servers, but maybe the mellanox nics have special drivers that are breaking?02:59
*** john-davidge has joined #openstack-infra03:04
*** hamzy has joined #openstack-infra03:06
*** john-davidge has quit IRC03:09
*** ijw has joined #openstack-infra03:09
*** mat128 has joined #openstack-infra03:11
kevinbentonfungi: hey, who is the best person to ask about the multinode networking topology?03:13
*** ijw has quit IRC03:13
kevinbentonfungi: (i.e. how nodes are connected together)03:13
mtreinishkevinbenton: clarkb is who I would ask about that03:14
kevinbentonmtreinish: ack03:14
mtreinishkevinbenton: also does http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/multinode_setup_info.txt have whatever you're looking for?03:14
clarkbkevinbenton: ya ^ is the doc03:15
clarkbkevinbenton: I can try to help now if you have questions for now03:15
kevinbentonclarkb: that doc doesn't quite have what i'm looking for03:15
clarkbcmurphy: thanks, I think I discovered how to recover so will work on better kernel install then try again03:15
kevinbentonclarkb: at least not for linuxbridge03:15
clarkbcmurphy: and hopefully won't have to recover off of the ilo again :)03:15
kevinbentonclarkb: http://logs.openstack.org/65/456665/1/check/gate-grenade-dsvm-neutron-linuxbridge-multinode-ubuntu-xenial-nv/74b2004/logs/worlddump-2017-04-13-194801.txt.gz03:15
kevinbentonclarkb: what i want to know is what eth0 and eth1 are03:16
clarkbkevinbenton: eth0 and eth1 are the cloud provided interfaces they are "real"03:16
*** tonytan4ever has quit IRC03:16
clarkbkevinbenton: in this case looks like rackspace. eth0 is the public interfacen and eth1 the private03:16
clarkbthey should not be used for testing generally03:16
*** tonytan4ever has joined #openstack-infra03:16
kevinbentonclarkb: well the reason i ask is that we're getting occasional failures on rax03:17
kevinbentonclarkb: with linux bridge multinode03:17
kevinbentonclarkb: and i think it's due to a failure to carry multicast traffic03:17
*** esikache1 has joined #openstack-infra03:17
kevinbentonclarkb: eth1 between the two nodes are on different networks03:17
kevinbentonclarkb: so it honestly doesn't surprise me that multicast routing may be broken between the two03:18
clarkbyou can't generally do multicast thats right03:18
clarkbwhcih is why you shouldn't test over eth0 and eth103:18
clarkbyou should use overlays if you need that kind of control03:18
kevinbentonclarkb: well OVS does utilize it for it's vxlan meshes03:18
kevinbentonclarkb: the 'local_ip' option the neutron agent03:19
kevinbentonclarkb: but it's direct unicast03:19
kevinbentonclarkb: so nothing magical03:19
clarkbI'm not sure I understand that03:19
kevinbentonOVS case: node1 and node2 are running OVS agents03:20
kevinbentonthey are both configured with a local_ip that gets passed between them via the neutron server for forming a VXLAN tunnel mesh03:20
mtreinishpabelanger: is this something youve seen before? http://logs.openstack.org/11/456811/1/check/gate-openstackci-beaker-centos-7/dcb57e5/console.html#_2017-04-14_02_37_33_91243903:21
clarkbkevinbenton: and where is multicast coming in?03:21
kevinbentonso they both establish tunnels with each other as endpoints03:21
kevinbentonso multicast comes in in the linux bridge case03:21
kevinbentonwithout L2pop enabled (which it's not here)03:21
kevinbentonlinux bridge sends traffic to unlearned destinations to a multicast address03:22
*** esikache1 has quit IRC03:22
clarkbright so for that you likely want an overlay like we do for br-ex and the nova net case?03:22
kevinbentonit depends on how hard that would be :)03:22
clarkbthen you can multicast over that or just have shared l2 and everyone is happy (we actually rely on shared l2 happyness in most of the cases I think)03:23
clarkbits already built in03:23
kevinbentonif that's a lot of work, we can just enable l2_population for linux bridge and call it a day03:23
kevinbentonoh okay03:23
clarkbdevstack-gate has a function called setup-vxlan-overlay or something03:23
clarkbyou just have to do that and you automagically get all the subnodes hanging off the primary node using ovs + vxlan03:24
*** nicolasbock has quit IRC03:24
kevinbentonin the linuxbridge case i don't think we even have ovs installed03:24
clarkbthe test env does03:24
clarkbfor exactly this purpose03:24
kevinbentonah, i saw this in worlddump "*** ovs-vsctl not found: skipping"03:25
clarkbbasically since we can't rely on the underlying cloud to do this for us we provide our own virtual switch and cabling for every test case that needs it :)03:25
clarkbya its likely that the job currently doesn't opt into it03:25
kevinbentonok03:25
clarkbbut all the framework for doing so is there you just have to turn it on basically03:25
*** esikache1 has joined #openstack-infra03:25
kevinbentonwhere do i put the logic that says to load this stuff up for the linux bridge multinode job03:25
clarkbkevinbenton: let me find links into code03:26
clarkbhttps://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/devstack-vm-gate.sh#n142 probably right around there03:26
clarkbright now its nova net or dvr because those are the only two cases that have needed it03:27
kevinbentonclarkb: ack. i'll look into this tonight03:28
kevinbentonclarkb: thanks for the pointer03:28
openstackgerritXingchao Yu proposed openstack-infra/project-config master: New project - Foxnut  https://review.openstack.org/45649403:32
*** esikache1 has quit IRC03:32
*** hongbin has joined #openstack-infra03:33
*** hongbin has quit IRC03:33
*** hongbin has joined #openstack-infra03:34
*** kzaitsev_mb has joined #openstack-infra03:34
*** jascott1 has quit IRC03:36
*** jascott1 has joined #openstack-infra03:36
*** armax has joined #openstack-infra03:37
*** kzaitsev_mb has quit IRC03:40
*** ricardoas has joined #openstack-infra03:40
*** jascott1 has quit IRC03:41
*** harlowja has joined #openstack-infra03:44
cmurphyclarkb: btw iirc vanilla has the mellanox nics but the other regions don't, so it that's the issue then the others shouldn't have the same problem03:52
cmurphykind of scary to test that though03:52
*** mat128 has quit IRC03:53
*** ricardoas has quit IRC03:55
*** esikache1 has joined #openstack-infra03:58
*** beslemon_ has quit IRC04:00
*** zhurong has quit IRC04:02
*** esikache1 has quit IRC04:03
*** ricardoas has joined #openstack-infra04:04
*** faizy has joined #openstack-infra04:05
*** csomerville has joined #openstack-infra04:10
*** cody-somerville has quit IRC04:13
*** Julien-zte has quit IRC04:14
*** Julien-z_ has joined #openstack-infra04:14
*** Julien-z_ has quit IRC04:15
*** nadya has joined #openstack-infra04:22
*** gongysh has joined #openstack-infra04:23
*** harlowja has quit IRC04:23
*** esikache1 has joined #openstack-infra04:25
*** psachin has joined #openstack-infra04:25
*** hongbin has quit IRC04:27
*** zhurong has joined #openstack-infra04:29
*** esikache1 has quit IRC04:30
*** gongysh has quit IRC04:31
*** kzaitsev_mb has joined #openstack-infra04:36
*** harlowja has joined #openstack-infra04:38
*** kzaitsev_mb has quit IRC04:41
*** thorst has joined #openstack-infra04:45
*** prateek has joined #openstack-infra04:53
linziI follow the steps in website : https://docs.openstack.org/infra/openstackci/third_party_ci.html#start-nodepool05:02
linziCould anyone tell me how to fix it?    Thank you     https://thepasteb.in/p/48hYgZ3q5BGHR05:02
linziI have copied the entire contents of "https://github.com/openstack-infra/project-config/tree/master/nodepool/elements" and "https://github.com/openstack-infra/project-config/tree/master/nodepool/script"05:02
linziand not modified any file.05:02
linziIt still cannot work...05:03
*** rcernin has quit IRC05:08
*** set-an has joined #openstack-infra05:12
*** set-an has left #openstack-infra05:12
*** baoli has quit IRC05:13
*** thorst has quit IRC05:14
*** srobert has joined #openstack-infra05:15
*** rcernin has joined #openstack-infra05:17
*** gcb has quit IRC05:19
*** srobert has quit IRC05:20
*** MasterOfBugs has quit IRC05:21
*** MasterOfBugs has joined #openstack-infra05:21
*** esikache1 has joined #openstack-infra05:24
*** MasterOfBugs has quit IRC05:24
*** MasterOfBugs has joined #openstack-infra05:24
*** prateek has quit IRC05:26
*** john-davidge has joined #openstack-infra05:31
*** gcb has joined #openstack-infra05:32
*** pramodrj07 has joined #openstack-infra05:32
*** pramodrj07 has quit IRC05:35
*** MasterOfBugs has quit IRC05:35
*** MasterOfBugs has joined #openstack-infra05:35
*** john-davidge has quit IRC05:36
*** alexchadin has joined #openstack-infra05:37
*** kzaitsev_mb has joined #openstack-infra05:38
*** hichihara has joined #openstack-infra05:40
*** prateek has joined #openstack-infra05:41
*** kzaitsev_mb has quit IRC05:44
*** alexchadin has quit IRC05:46
*** nadya has quit IRC05:49
*** masayukig has quit IRC05:49
*** masayukig has joined #openstack-infra05:49
*** jascott1 has joined #openstack-infra05:53
*** harlowja has quit IRC05:55
*** cshastri has joined #openstack-infra05:55
*** esikache1 has quit IRC05:59
*** gongysh has joined #openstack-infra06:00
*** nadya has joined #openstack-infra06:09
*** thorst has joined #openstack-infra06:11
*** Hal has quit IRC06:12
*** bhavik1 has joined #openstack-infra06:15
openstackgerritXingchao Yu proposed openstack-infra/project-config master: New project - Foxnut  https://review.openstack.org/45649406:16
*** thorst has quit IRC06:17
*** Julien-zte has joined #openstack-infra06:17
*** prateek has quit IRC06:20
*** zhurong has quit IRC06:20
*** armax has quit IRC06:21
*** nadya has quit IRC06:23
openstackgerritMerged openstack-dev/pbr master: doc: Document Sphinx integration  https://review.openstack.org/45615406:25
*** esikache1 has joined #openstack-infra06:26
*** zhurong has joined #openstack-infra06:32
*** bhavik1 has quit IRC06:37
*** Julien-zte has quit IRC06:39
*** Julien-zte has joined #openstack-infra06:39
*** kzaitsev_mb has joined #openstack-infra06:40
*** armax has joined #openstack-infra06:41
ttxclarkb: odsreg.o.o is good to go, I don't need it anymore06:42
*** Julien-zte has quit IRC06:42
*** Julien-zte has joined #openstack-infra06:43
*** ralonsoh has joined #openstack-infra06:45
*** kzaitsev_mb has quit IRC06:46
*** esikache1 has quit IRC06:47
*** john-davidge has joined #openstack-infra06:47
*** andreas-f has quit IRC06:51
*** john-davidge has quit IRC06:52
*** amotoki has joined #openstack-infra07:01
*** Julien-zte has quit IRC07:07
*** Julien-zte has joined #openstack-infra07:07
*** HeOS_ has joined #openstack-infra07:08
*** jascott1 has quit IRC07:09
*** jascott1 has joined #openstack-infra07:09
*** tesseract has joined #openstack-infra07:10
*** hichihara has quit IRC07:11
*** thorst has joined #openstack-infra07:13
*** jascott1 has quit IRC07:13
*** esikache1 has joined #openstack-infra07:15
*** srobert has joined #openstack-infra07:17
*** thorst has quit IRC07:18
*** esberglu has joined #openstack-infra07:18
*** srobert has quit IRC07:21
*** esberglu has quit IRC07:22
*** Hal has joined #openstack-infra07:27
*** MasterOfBugs has quit IRC07:29
*** Julien-zte has quit IRC07:32
*** Julien-zte has joined #openstack-infra07:32
*** bauzas is now known as bauwser07:33
*** hashar has joined #openstack-infra07:34
*** nadya has joined #openstack-infra07:34
*** nadya has quit IRC07:39
*** nadya has joined #openstack-infra07:40
*** kzaitsev_mb has joined #openstack-infra07:42
*** john-davidge has joined #openstack-infra07:42
*** hashar_ has joined #openstack-infra07:44
*** hashar has quit IRC07:44
*** kzaitsev_mb has quit IRC07:47
*** john-davidge has quit IRC07:47
*** chkumar|off is now known as chandankumar07:48
*** alexchadin has joined #openstack-infra07:51
*** flepied has quit IRC07:51
*** hashar_ has quit IRC07:54
*** Julien-zte has quit IRC07:56
*** Julien-zte has joined #openstack-infra07:57
*** hashar has joined #openstack-infra07:58
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-infra08:01
*** andreas-f has joined #openstack-infra08:01
*** linzi has quit IRC08:02
*** linzi has joined #openstack-infra08:02
*** Julien-zte has quit IRC08:05
*** Julien-zte has joined #openstack-infra08:05
*** thorst has joined #openstack-infra08:14
*** rossella_s has joined #openstack-infra08:15
*** flepied has joined #openstack-infra08:17
*** efoley has joined #openstack-infra08:18
*** mhickey has joined #openstack-infra08:18
*** yuanying has joined #openstack-infra08:19
*** efoley_ has joined #openstack-infra08:20
*** yuanying_ has quit IRC08:21
*** askb has joined #openstack-infra08:22
*** tphummel has quit IRC08:23
*** efoley has quit IRC08:24
*** armax has quit IRC08:26
*** armax has joined #openstack-infra08:27
*** thorst has quit IRC08:33
*** tosky has joined #openstack-infra08:37
*** electrofelix has joined #openstack-infra08:39
*** kzaitsev_mb has joined #openstack-infra08:44
*** kzaitsev_mb has quit IRC08:50
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: neutron-vpnaas: Move tempest job from experimental to non-voting  https://review.openstack.org/45301508:50
*** liusheng has quit IRC09:12
*** markvoelker_ has quit IRC09:17
*** srobert has joined #openstack-infra09:17
openstackgerritAnton Chevychalov proposed openstack-infra/project-config master: Enable EPEL for bindep-fallback.txt  https://review.openstack.org/45686709:18
*** amotoki has quit IRC09:20
*** srobert has quit IRC09:22
*** e0ne has joined #openstack-infra09:26
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Run ovb jobs in transition mode  https://review.openstack.org/45413409:29
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test running OVB ha/nonha job with featureset  https://review.openstack.org/45427609:30
*** Julien-zte has quit IRC09:30
*** gcb has quit IRC09:31
*** links has quit IRC09:33
*** liusheng has joined #openstack-infra09:38
*** lewo has quit IRC09:39
*** lewo has joined #openstack-infra09:40
*** aviau has quit IRC09:44
*** gcb has joined #openstack-infra09:44
*** efoley_ is now known as efoley09:44
*** aviau has joined #openstack-infra09:44
*** e0ne has quit IRC09:46
*** kzaitsev_mb has joined #openstack-infra09:46
*** ijw has joined #openstack-infra09:51
*** kzaitsev_mb has quit IRC09:53
*** ijw has quit IRC09:55
*** e0ne has joined #openstack-infra09:56
*** xarses_ has quit IRC10:00
*** nadya has quit IRC10:03
*** john-davidge has joined #openstack-infra10:03
*** zhurong has quit IRC10:03
*** liujiong has quit IRC10:07
*** alexchadin has quit IRC10:12
*** ggnel_t has joined #openstack-infra10:12
ggnel_tyo10:12
*** markvoelker has joined #openstack-infra10:18
ssbarneaAny git-review expert around here? I tried to enable it for gerrit's gerrit instance but I was not able to succeed yet : https://storyboard.openstack.org/#!/story/200099110:20
*** ggnel_t has quit IRC10:21
*** nicolasbock has joined #openstack-infra10:23
*** markvoelker has quit IRC10:23
*** yamahata has quit IRC10:24
*** thorst has joined #openstack-infra10:30
*** amotoki has joined #openstack-infra10:31
*** gcb has quit IRC10:34
*** thorst has quit IRC10:35
*** nadya has joined #openstack-infra10:40
*** kzaitsev_mb has joined #openstack-infra10:48
*** yanyanhu has quit IRC10:51
*** kzaitsev_mb has quit IRC10:53
*** linzi has quit IRC11:00
*** chlong has joined #openstack-infra11:02
*** amotoki_ has joined #openstack-infra11:04
*** amotoki has quit IRC11:07
*** ramishra_ has quit IRC11:13
*** sdague has joined #openstack-infra11:15
*** srobert has joined #openstack-infra11:18
*** jkilpatr has quit IRC11:19
*** chlong has quit IRC11:20
*** gongysh has quit IRC11:21
*** srobert has quit IRC11:23
*** yamamoto has quit IRC11:25
*** ggnel_t has joined #openstack-infra11:27
*** spotz_ is now known as spotz11:32
*** john-davidge has quit IRC11:38
*** ggnel_t has quit IRC11:41
*** ggnel_t has joined #openstack-infra11:41
*** thorst has joined #openstack-infra11:42
*** amotoki_ has quit IRC11:46
*** amotoki has joined #openstack-infra11:46
*** rhallisey has joined #openstack-infra11:47
*** kzaitsev_mb has joined #openstack-infra11:50
*** kzaitsev_mb has quit IRC11:56
*** faizy_ has joined #openstack-infra11:56
*** jamesdenton has quit IRC11:58
*** esikache1 has quit IRC11:59
*** amotoki has quit IRC11:59
*** faizy has quit IRC12:00
*** jkilpatr has joined #openstack-infra12:01
*** Hal has quit IRC12:01
*** faizy has joined #openstack-infra12:03
*** faizy_ has quit IRC12:06
*** hashar has quit IRC12:08
*** ssbarnea has quit IRC12:13
*** Hal has joined #openstack-infra12:18
*** hashar has joined #openstack-infra12:19
*** markvoelker has joined #openstack-infra12:20
*** efoley has quit IRC12:21
*** markvoelker has quit IRC12:24
*** markvoelker has joined #openstack-infra12:25
*** gouthamr has joined #openstack-infra12:26
*** faizy_ has joined #openstack-infra12:31
pabelangermorning12:32
*** ildikov is now known as coffeecat12:32
*** coffeecat is now known as coffee_cat12:32
pabelangerclarkb: I released the locks on mirror-update.o.o this morning, we can re-grab them for afs upgrade12:33
pabelangeralso, won't be around for the morning but around lunch time12:33
*** faizy has quit IRC12:35
*** yamamoto has joined #openstack-infra12:39
*** askb has quit IRC12:44
*** yamamoto has quit IRC12:44
*** fguillot has joined #openstack-infra12:47
*** Guest52586 has quit IRC12:50
*** kzaitsev_mb has joined #openstack-infra12:52
*** faizy_ has quit IRC12:53
*** eroux has joined #openstack-infra12:56
*** ggnel_t has quit IRC12:56
*** kzaitsev_mb has quit IRC12:58
*** hashar has quit IRC12:58
*** zigo has joined #openstack-infra12:58
*** jaypipes is now known as feeblepipes12:59
*** signed8bit_Zzz is now known as signed8b_13:05
*** xyang1 has joined #openstack-infra13:05
openstackgerritRosario Di Somma proposed openstack-infra/shade master: WIP: Use REST for the cinder volumes list  https://review.openstack.org/45673613:05
tusharPabelanger : good morning13:13
*** mhickey has quit IRC13:14
*** dasm has left #openstack-infra13:15
tusharPabelanger : Can I skip the UCA part in CI ? I mean commenting the "sudo add-apt-repository -y cloud-archive:ocata"13:16
tusharstill having issue with this in our CI setup13:17
*** mdrabe has joined #openstack-infra13:17
*** baoli has joined #openstack-infra13:18
*** srobert has joined #openstack-infra13:19
*** esberglu has joined #openstack-infra13:19
*** nadya has quit IRC13:22
*** hashar has joined #openstack-infra13:23
*** srobert has quit IRC13:24
*** esberglu has quit IRC13:24
*** rderose has joined #openstack-infra13:25
*** yamamoto has joined #openstack-infra13:31
*** yamamoto has quit IRC13:32
*** yamamoto has joined #openstack-infra13:32
*** tonytan_brb has joined #openstack-infra13:32
*** yamamoto_ has joined #openstack-infra13:33
*** rderose has quit IRC13:34
*** tonytan4ever has quit IRC13:35
*** Dinesh_Bhor has quit IRC13:36
*** yamamoto has quit IRC13:36
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921513:36
*** wolverineav has joined #openstack-infra13:38
*** persia has joined #openstack-infra13:45
*** tesseract has quit IRC13:45
*** pilgrimstack has joined #openstack-infra13:46
*** tesseract has joined #openstack-infra13:46
*** esberglu has joined #openstack-infra13:47
*** jamesdenton has joined #openstack-infra13:47
*** gongysh has joined #openstack-infra13:48
*** rbrndt_ has joined #openstack-infra13:51
*** gongysh has quit IRC13:55
*** kzaitsev_mb has joined #openstack-infra13:55
*** xarses has joined #openstack-infra13:57
*** yamamoto_ has quit IRC13:57
*** yamamoto has joined #openstack-infra13:57
*** kzaitsev_mb has quit IRC14:00
*** larainema has quit IRC14:02
*** yamamoto has quit IRC14:02
*** larainema has joined #openstack-infra14:02
*** anton_chevychalo has joined #openstack-infra14:02
*** xarses has quit IRC14:02
*** armax has quit IRC14:03
*** Julien-zte has joined #openstack-infra14:03
anton_chevychaloHey there! Could someone please take a look at https://review.openstack.org/#/c/456867/14:04
anton_chevychaloAfter EPEL enablement in I714cd44afb77191c89ac82f79cf57aa85a794e6d we have a troubles with bindep-fallback.txt14:05
*** xyang1 has quit IRC14:11
*** xyang1 has joined #openstack-infra14:11
*** armax has joined #openstack-infra14:13
*** tphummel has joined #openstack-infra14:17
*** felipemonteiro has joined #openstack-infra14:20
*** dimtruck is now known as zz_dimtruck14:21
*** mortenie has joined #openstack-infra14:23
*** nadya has joined #openstack-infra14:29
*** EricGonczer_ has joined #openstack-infra14:30
*** rbrndt_ has quit IRC14:30
*** rbrndt has joined #openstack-infra14:30
*** esikache1 has joined #openstack-infra14:32
*** EricGonc_ has joined #openstack-infra14:32
*** EricGonczer_ has quit IRC14:35
*** annegentle has joined #openstack-infra14:35
*** cshastri has quit IRC14:37
*** Kevin_Zheng has quit IRC14:37
*** Kevin_Zheng has joined #openstack-infra14:37
*** tphummel has quit IRC14:39
*** xarses has joined #openstack-infra14:42
*** zz_dimtruck is now known as dimtruck14:43
*** sdague has quit IRC14:50
*** scottda has joined #openstack-infra14:52
*** dmsimard is now known as dmsimard|afk14:55
*** flepied has quit IRC14:58
*** yamamoto has joined #openstack-infra14:59
*** xarses_ has joined #openstack-infra14:59
*** andreww has joined #openstack-infra15:00
*** esikache1 has quit IRC15:00
*** HeOS_ has quit IRC15:02
*** mortenie has quit IRC15:03
*** lin_yang has quit IRC15:03
*** lin_yang has joined #openstack-infra15:03
openstackgerritBogdan Dobrelya proposed openstack-infra/tripleo-ci master: Adapt getthelogs UX for more use cases  https://review.openstack.org/44955215:04
*** xarses_ has quit IRC15:04
*** ijw has joined #openstack-infra15:06
*** knangia has joined #openstack-infra15:07
*** Julien-zte has quit IRC15:10
*** Julien-zte has joined #openstack-infra15:11
*** Julien-zte has quit IRC15:16
*** Julien-zte has joined #openstack-infra15:16
*** Julien-zte has quit IRC15:16
*** jamesdenton has quit IRC15:17
*** Julien-zte has joined #openstack-infra15:17
clarkbtushar: when I wrote the chagne to enable it in devstack I decided to not make it configurable beacuse we knew the base versions weren't working well15:17
*** burgerk has joined #openstack-infra15:17
clarkbtushar: are you still having problems now that pabelanger updated it to use the helper function for apt-get update?15:17
*** mortenie has joined #openstack-infra15:20
*** inc0 has joined #openstack-infra15:21
*** Julien-zte has quit IRC15:22
clarkbpabelanger: ok, ping when you are back and we can pick up afs things again. In the meantime I am going to fiddle with the baremetal server to see if I can't sort out what was wrong with its kernel update15:22
*** Qiming has quit IRC15:22
*** Julien-zte has joined #openstack-infra15:22
*** Julien-zte has quit IRC15:27
*** tomhambleton_ has quit IRC15:27
*** tomhambleton_ has joined #openstack-infra15:27
*** Julien-zte has joined #openstack-infra15:28
clarkbanton_chevychalo: reviewed, let me know what you think15:28
*** Qiming has joined #openstack-infra15:30
clarkbtushar: happy to help you work through the problem if you can provide more info15:30
*** ssbarnea has joined #openstack-infra15:31
*** ssbarnea has quit IRC15:33
*** mortenie has quit IRC15:34
*** ssbarnea has joined #openstack-infra15:35
anton_chevychaloclarkb: You are right: I have to do something to restrict that to rpm based distros. But as far as I know it is forbiden to change bindep-fallback.txt15:36
*** nadya has quit IRC15:37
*** rcernin has quit IRC15:37
*** rbrndt has quit IRC15:37
*** jordanP_ has joined #openstack-infra15:38
*** ssbarnea has quit IRC15:38
jordanP_clarkb, if you have a chance, could you review https://review.openstack.org/#/c/454738/ (Swift: don't start optionnal services by default) please ?15:39
clarkbanton_chevychalo: I think that we can modify bindep-fallback here just for centos and python3415:39
clarkbanton_chevychalo: since its currently not working anyways and most projects have already said they don't epel or will opt into it themselves implying no one is using python34 on centos. If there is anyone they can opt in like the others?15:40
openstackgerritAnton Chevychalov proposed openstack-infra/project-config master: Enable EPEL for bindep-fallback.txt  https://review.openstack.org/45686715:41
anton_chevychaloclarkb: Well I am stranger there. There is no needs python for jobs I have. So I can do that patch.15:42
*** EricGonc_ has quit IRC15:43
*** EricGonczer_ has joined #openstack-infra15:43
*** ijw has quit IRC15:44
clarkbjordanP_: pulling it up now15:46
clarkbanton_chevychalo: ping me when the patch to remove it from fallback list is up, will make sure to review it15:46
jordanP_thanks15:47
clarkbjordanP_: my only concern with that change is the swift functional test job. I'm not sure if they rely on all the services there, but maybe you can explicitly enable them in that one job and depends on that change in the current change?15:48
pabelangerjust coming back online now15:48
jordanP_clarkb, I rant the swift func test with that change, they pass15:48
pabelangerlet me grab a coffee and catch up15:48
jordanP_*ran15:48
clarkbjordanP_: oh interesting, I guess they don't test the additional services there then?15:49
jordanP_they don't, not in the func tests. They have something call "probe" tests that might exercice them but there's no job for that tox environment yet, and no Jenkins job associated15:50
clarkbjordanP_: gotcha, thanks for clearing that up. I will approve the change15:50
AJaeger_pabelanger, clarkb : Seeing the EPEL py34 comments, do we want to remove py34 from most jobs due to mitaka EOL first? https://review.openstack.org/#/c/456340/15:51
clarkbAJaeger_: I think most (maybe all) of those are using trusty py34 so somewhat orthogonal15:52
clarkbI think we can remove from centos now in fallback list to fix jobs then cleanup the rest of py34 when trusty is retired15:52
openstackgerritAnton Chevychalov proposed openstack-infra/project-config master: Remove python34-devel from bindep-fallback.txt  https://review.openstack.org/45686715:54
pabelangerya, if we land removal of python34 from bindep-fallback.txt, we can roll new images15:56
clarkbcmurphy: mnaser rcarrillocruz Ok clearly I just needed to sleep on it. But I differed the package installs between baremetal00 and compute000 and found that linux-image-generic was missing on baremeta00 as well as the -extras kernel package which likely has modules i needed. Got both of those installed, rebooted and I think our general package update process should properly install new kernels now.15:57
clarkbThanks for the help15:57
mnaserthat was probably it, glad it's all worked out clarkb :)15:57
AJaeger_clarkb: ah, I see - you're talking CentOS while my change covers Ubuntu mainly. Ok.15:57
*** psachin has quit IRC15:57
clarkbAJaeger_: yup15:57
*** kzaitsev_mb has joined #openstack-infra15:58
clarkbanton_chevychalo: +2 thanks15:58
*** e0ne has quit IRC15:59
anton_chevychaloclarkb: Thanks to you.16:01
*** mdrabe has quit IRC16:02
pabelanger+316:02
pabelangeronce it lands, I'll start new image builds16:03
*** kzaitsev_mb has quit IRC16:03
*** ralonsoh has quit IRC16:04
AJaeger_pabelanger: could you put a small spring cleanup on your review queue, please? https://review.openstack.org/#/c/455795/ contains some unused scripts in project-config...16:04
*** gyee has joined #openstack-infra16:05
*** pilgrimstack has quit IRC16:07
*** hongbin has joined #openstack-infra16:07
*** mdrabe has joined #openstack-infra16:08
openstackgerritMerged openstack-infra/project-config master: Remove python34-devel from bindep-fallback.txt  https://review.openstack.org/45686716:09
*** csomerville has quit IRC16:09
*** csomerville has joined #openstack-infra16:10
*** yamamoto has quit IRC16:10
*** larainema has quit IRC16:12
*** mwhahaha has quit IRC16:14
*** hashar has quit IRC16:14
*** mwhahaha has joined #openstack-infra16:15
*** mhickey has joined #openstack-infra16:15
*** Apoorva has joined #openstack-infra16:16
*** rbrndt has joined #openstack-infra16:20
*** armax has quit IRC16:21
*** armax has joined #openstack-infra16:21
anton_chevychaloWhat is typical time lag for zuul to accept merged configs?16:22
openstackgerritSaad Zaher proposed openstack-infra/project-config master: Update freezer and freezer-api gate jobs  https://review.openstack.org/45673816:22
*** tbarron has quit IRC16:23
clarkbanton_chevychalo: in this case we will have to rebuild the centos image and get it uploaded into all the cloud regions so it will vary by cloud16:23
clarkbanton_chevychalo: some accept uploads much more quickly than others, probably a few hours16:23
anton_chevychaloOk. Got it.16:24
clarkbanton_chevychalo: pabelanger and I have one more server to patch then we can make sure that build and upload are queued up properly16:24
*** tbarron has joined #openstack-infra16:24
*** reed has quit IRC16:27
*** reed has joined #openstack-infra16:29
*** amotoki has joined #openstack-infra16:31
*** pbourke has quit IRC16:33
tushar_clarkb : with  helper function also devstack installation is failing16:33
*** Julien-zte has quit IRC16:34
clarkbtushar_: can you provide logs or link to them? I'm curious to see what is failing16:34
*** Hal has quit IRC16:34
*** pbourke has joined #openstack-infra16:34
szaherAJaeger_: https://review.openstack.org/#/c/456738/ Done !, Thanks for the help16:34
openstackgerritJimmy McCrory proposed openstack-infra/project-config master: Enable EPEL before installing distro packages  https://review.openstack.org/45672016:36
tushar_clarkb : https://pastebin.com/qNYeEKRT16:36
tushar_we need to change anything ..after addition of helper function16:37
tushar_?16:37
*** sripriya has quit IRC16:38
*** sripriya has joined #openstack-infra16:38
clarkbtushar_: can you hit the cloud archive url from that host?16:38
clarkbtushar_: try using wget or curl manually and see what you get16:38
tushar_clarkb : https://pastebin.com/NB9pJVsw16:41
tushar_unable to get response16:42
clarkbtushar_: ok, are you sure your proxy is configured properly?16:42
*** jascott1 has joined #openstack-infra16:43
tushar_clarkb : yes. Even I can connect to ubuntu archive16:44
clarkbtushar_: can you test that using wget too?16:44
tushar_yes already tested16:45
tushar_https://pastebin.com/KtsUn7T816:46
clarkbtushar_: I would check your proxy configuration to make sure it allows connections to cloud archive and if so check the logs to see why the proxy couldn't connect16:47
pabelangeranton_chevychalo: clarkb: AJaeger_: centos-7 DIB building now16:47
*** Kevin_Zheng has quit IRC16:47
*** robcresswell has quit IRC16:48
*** robcresswell has joined #openstack-infra16:48
*** dtantsur|afk is now known as dtantsur16:54
*** hongbin has quit IRC16:58
*** jascott1 has quit IRC17:00
*** kzaitsev_mb has joined #openstack-infra17:01
*** anton_chevychalo has quit IRC17:03
*** e0ne has joined #openstack-infra17:04
*** rossella_s has quit IRC17:04
*** burgerk has quit IRC17:04
*** kzaitsev_mb has quit IRC17:06
tushar_clarkb : I think its because of privoxy... I need to add cloudarchive url in the list17:06
openstackgerritMerged openstack-infra/devstack-gate master: Swift: don't start optionnal services by default  https://review.openstack.org/45473817:09
dtantsurhi folks! is there any form of reservation for rooms @ https://wiki.openstack.org/wiki/Infrastructure/Conferencing ?17:09
clarkbdtantsur: we've mostly just done it ad hoc. Haven't heard of any problems with collisions17:10
*** jordanP_ has quit IRC17:10
clarkbdtantsur: there is a sprints page on the wiki that I think people have used to reserve the sprints irc channel and then also a pbx room17:10
pabelangerya, we have a bunch of rooms setup17:10
dtantsuryeah, I'm editing the sprints page, but I guess we'll use our channel17:10
*** flepied has joined #openstack-infra17:11
*** pilgrimstack has joined #openstack-infra17:13
openstackgerritkevin raymond proposed openstack-infra/git-review master: Fix unused warn function argument.  https://review.openstack.org/45692917:14
openstackgerritkevin raymond proposed openstack-infra/git-review master: Define a new nothin option argument to fine tune git-send-pack.  https://review.openstack.org/45693017:14
openstackgerritkevin raymond proposed openstack-infra/git-review master: Prevent rebase against a non existing remote folder.  https://review.openstack.org/45693117:14
*** esberglu has quit IRC17:14
cmurphyclarkb: yay i'm glad it's working17:14
*** pilgrimstack has left #openstack-infra17:14
*** ssbarnea has joined #openstack-infra17:16
*** ssbarnea has quit IRC17:19
*** ssbarnea has joined #openstack-infra17:22
*** e0ne has quit IRC17:22
*** dtantsur is now known as dtantsur|afk17:23
pabelanger#status log vos release npm.mirror --localauth currently running from screen in afsdb0117:23
openstackstatuspabelanger: finished logging17:23
*** esberglu has joined #openstack-infra17:25
*** nadya has joined #openstack-infra17:25
*** flepied has quit IRC17:26
*** ijw has joined #openstack-infra17:31
*** armax_ has joined #openstack-infra17:31
*** erikmwilson has quit IRC17:32
pabelangeranton_chevychalo: clarkb: AJaeger_: centos-7 uploads started17:32
*** tesseract has quit IRC17:33
*** armax has quit IRC17:35
*** armax_ is now known as armax17:35
*** electrofelix has quit IRC17:40
*** yamahata has joined #openstack-infra17:41
*** nadya has quit IRC17:49
*** ssbarnea has quit IRC17:50
*** ssbarnea has joined #openstack-infra17:52
openstackgerritkevin raymond proposed openstack-infra/git-review master: Warn which patchset are going to be obsoleted.  https://review.openstack.org/45693717:52
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add AFS maintenance docs  https://review.openstack.org/45693817:53
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add kerberos maintenance docs  https://review.openstack.org/45693917:53
clarkbpabelanger: ^ how do those look?17:53
*** signed8b_ is now known as signed8bit_Zzz17:54
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Remove old clouds from clouds_layouts.yml  https://review.openstack.org/45694117:55
rcarrillocruzclarkb: sorry, was travelling yesterday back from US and went straight to vaca17:55
rcarrillocruzAll sorted?17:55
*** tbarron has quit IRC17:55
clarkbrcarrillocruz: yup all sorted, Seemed to be a problem with the base image in how its kernel was installed. I managed to use console to boot old kernel that worked, then diffed against a compute host to figure out how the kernel needed changing then got it rebooted and happy17:56
*** ijw has quit IRC17:57
pabelangerclarkb: https://review.openstack.org/#/c/456755/ should fix our system-config failures17:58
clarkbrcarrillocruz: I just needed to get some sleep then the path forward was much clearer to me. Thanks again everyone for helping17:58
pabelangerclarkb: also, left commands on 45693817:58
rcarrillocruzYup17:58
clarkbpabelanger: thanks, I'll edit then push up once the system-config failures fix merges17:59
dirkreview for https://review.openstack.org/#/c/456743/ highly appreciated - we need to get the global requirements proposal bot running again..18:00
*** xarses has quit IRC18:00
*** tbarron has joined #openstack-infra18:00
*** knangia has quit IRC18:01
*** tbarron has quit IRC18:01
*** sdague has joined #openstack-infra18:02
clarkbdirk: reviewed, one small thing18:02
openstackgerritMerged openstack-infra/puppet-openstackci master: Fix beaker-rspec tests for centos7  https://review.openstack.org/45675518:02
*** kzaitsev_mb has joined #openstack-infra18:02
*** baoli has quit IRC18:04
openstackgerritStephen Finucane proposed openstack-dev/pbr master: docs: Use definition lists  https://review.openstack.org/45615518:04
*** baoli has joined #openstack-infra18:05
openstackgerritStephen Finucane proposed openstack-dev/pbr master: Stop building man pages by default  https://review.openstack.org/45615618:05
*** oanson has quit IRC18:05
openstackgerritStephen Finucane proposed openstack-dev/pbr master: Remove 'build_sphinx_latex'  https://review.openstack.org/45615718:05
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add AFS maintenance docs  https://review.openstack.org/45693818:07
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add kerberos maintenance docs  https://review.openstack.org/45693918:07
clarkbpabelanger: that better ^?18:07
*** tbarron has joined #openstack-infra18:07
*** oanson has joined #openstack-infra18:07
*** kzaitsev_mb has quit IRC18:08
*** tphummel has joined #openstack-infra18:08
openstackgerritDirk Mueller proposed openstack-infra/system-config master: Install libsystemd-{journal}-dev on proposal slave  https://review.openstack.org/45674318:09
dirkclarkb: thanks, should be better now18:10
*** signed8bit_Zzz is now known as signed8b_18:10
*** harlowja has joined #openstack-infra18:12
pabelangerclarkb: one more round of comments, but looks good18:12
*** tphummel has quit IRC18:12
clarkbpabelanger: for the auxiliary status lines, is that in addition to what is already printed there or a replacement?18:14
*** jkilpatr has quit IRC18:15
clarkblooks like reaplcement based on irc scrollback, thanks18:16
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add AFS maintenance docs  https://review.openstack.org/45693818:16
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add kerberos maintenance docs  https://review.openstack.org/45693918:16
*** armax has quit IRC18:18
*** MasterOfBugs has joined #openstack-infra18:20
*** tphummel has joined #openstack-infra18:20
openstackgerritMerged openstack-infra/project-config master: Enable EPEL before installing distro packages  https://review.openstack.org/45672018:20
gaieges! wondering how to use JJB with multibranch pipelines? i can't seem to find any documentation around it.  anyone know if this is possible?18:21
openstackgaieges: Error: "wondering" is not a valid command.18:21
clarkbzaro: zxiiro ^ do you know the answer to gaieges' question?18:21
zxiirogaieges: sorry i don't use pipelines18:22
zarogaieges: is that another type of job?18:22
zaropipeline jobs are supported, but not sure if that type is supported18:23
*** eroux has quit IRC18:23
gaiegeszaro: yeah .. mainly a multibranch job, with a pipeline in scm: https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Multibranch+Plugin18:23
zaroyeah, multibranch project is not there https://docs.openstack.org/infra/jenkins-job-builder/definition.html#modules18:25
gaiegesyeah i didnt see anything, was wondering if anyone had any luck with setting it up.  at the very least a pipeline-scm job would work but i can't seem to get that working either (i can give more details around that if it would help)18:26
*** portdirect has quit IRC18:26
*** portdirect has joined #openstack-infra18:26
zaroSorry, i haven’t used pipelines myself.18:29
*** krtaylor has quit IRC18:30
*** kzaitsev_mb has joined #openstack-infra18:32
*** ssbarnea has quit IRC18:33
*** ssbarnea has joined #openstack-infra18:34
*** clayg has quit IRC18:36
*** kzaitsev1mb has joined #openstack-infra18:37
*** ssbarnea has quit IRC18:37
pabelangerclarkb: ya, replacement18:38
*** ssbarnea has joined #openstack-infra18:39
*** kzaitsev_mb has quit IRC18:40
openstackgerritMerged openstack-infra/system-config master: Install libsystemd-{journal}-dev on proposal slave  https://review.openstack.org/45674318:42
clarkbdirk: ^ next puppet pulse should get that out to places18:42
dirkclarkb: thanks. /me hopes this will not fail18:43
dirkclarkb: any chance we could sneak in https://review.openstack.org/#/c/437154/ ?18:44
*** andreww is now known as xarses18:45
clarkbdirk: I've approved it18:45
clarkbout of curiousity do most people not fully type out zypper install? I guess my days of IOSing made me prefer using verbose strings rather than shortcuts (far easier to understand)18:46
dirkclarkb: happy to change that..18:47
clarkbdirk: nah its just a minor thing18:47
*** armax has joined #openstack-infra18:48
openstackgerritMerged openstack-infra/system-config master: Add support for openSUSE Leap  https://review.openstack.org/43715418:49
*** claudiub|2 has quit IRC18:50
*** cody-somerville has joined #openstack-infra18:51
*** cody-somerville has joined #openstack-infra18:51
openstackgerritDirk Mueller proposed openstack-infra/system-config master: suse: spell out zypper options in verbose fashion  https://review.openstack.org/45694618:51
dirkclarkb: ^^18:51
*** gaieges has quit IRC18:52
clarkbthanks18:52
*** csomerville has quit IRC18:53
openstackgerritDirk Mueller proposed openstack-infra/system-config master: Implement openSUSE slaves configuration  https://review.openstack.org/43888218:53
openstackgerritFélix Bouliane proposed openstack-infra/project-config master: Add ironic-pxe-snmp gate to virtualpdu  https://review.openstack.org/45694718:54
dirkclarkb: this is the last patch needed to have opensuse as nodepool images: https://review.openstack.org/#/c/438882/18:54
openstackgerritMike Perez proposed openstack-infra/project-config master: sphinx-feature-classification lib for features  https://review.openstack.org/45255618:55
clarkbdirk: thats mostly just to get the uid ranges to line up ya?18:55
*** krtaylor has joined #openstack-infra18:57
*** ac has joined #openstack-infra18:58
*** ac is now known as Guest2739718:58
*** Guest27397 is now known as anton_chev18:58
*** baoli has quit IRC18:59
dirkclarkb: yep, this is the diff: http://paste.openstack.org/show/606575/18:59
*** baoli has joined #openstack-infra18:59
pabelangerclarkb: dirk: we should hold off adding things to puppet, we'll likely remove them next week18:59
pabelangerif we can, lets just add opensuse without puppet19:00
pabelangerI don't think it will be much longer19:00
dirkpabelanger: which element would do login.defs then? I can't it or an open review to check out19:01
*** jbryce has quit IRC19:02
dirkI'm happy to test that the element works.. I'm just frustrated that I can't get relatively small patches in that are changing opensuse-only code that would allow to build an image that we can iterate on19:02
*** jbryce has joined #openstack-infra19:02
pabelangernodepool-base I would guess19:02
pabelangerneed to read up how we are using login.defs now19:03
clarkbI'm fine with adding to puppet now then cleaning those bits up as we go.19:03
clarkbbut if we want to jump ahead that fine too19:03
pabelangerya, I think it will be next week honest19:04
dirkpabelanger: can't find an open review that handles login.defs. does it exist?19:04
pabelangerdirk: no, that's what I am saying. We still need to write it19:04
*** ssbarnea has quit IRC19:04
pabelangerat the rate we are deleting puppet in system-config, I'd rather hold off on opensuse, until puppet is removed from DIB, then I'll help focus on bringing opensuse online19:05
openstackgerritFélix Bouliane proposed openstack-infra/project-config master: Add ironic-pxe-snmp gate to virtualpdu  https://review.openstack.org/45694719:05
pabelangerI don't think it will be much longer, maybe 2 weeks19:05
pabelangerat this point, we just need core reviewers for open reviews19:06
dirkpabelanger: I'm happy to work with you on this .. ;-)19:06
dirkpabelanger: its really just this one review though. literally. and then we could build an image19:06
clarkbnote that the login defs stuff only matters for our non test instances19:06
clarkbso that our uses (which we are removing) can have consistent uids19:07
pabelangerhttps://review.openstack.org/#/q/status:open+project:openstack-infra/system-config+topic:unpuppet-zuul-workers19:07
*** ssbarnea has joined #openstack-infra19:07
cmurphydirk: clarkb also need mt puppet-iptables patch, i'm on phone though so hard to link19:07
pabelangerclarkb: so, we nolonger need login.def once we remove embedded users?19:08
clarkbpabelanger: correct19:08
clarkbpabelanger: at least thats my understanding19:08
pabelangerk, then once we land nodepool changes, which are open reviews, we'll be able to stop installing users19:09
*** ihrachys has quit IRC19:09
dirkclarkb: pabelanger : the login.def was important so that the jenkins user has the proper uid (I think you're not removing that one?)19:09
clarkbdo we care what the jenkins uid is?19:09
clarkbthis became a thing with afs and making sure our uids lined up? maybe I should look at git logs19:10
dirkas a side effect19:10
pabelangerYa, don't think we need it for jenkins user in image19:10
pabelangergoing to look at history of patch19:10
dirkbuild was failing without the change. I can try to figure it out again if you want19:10
* dirk forgot the details19:10
*** hashar has joined #openstack-infra19:12
pabelangerya, we only pull in params.pp from users.pp now19:13
pabelangerso, once we remove that, they nolonger will be installed19:13
*** knangia has joined #openstack-infra19:15
dirkcmurphy: ah, this one.. https://review.openstack.org/#/c/447220/19:16
dirkcmurphy: is there an unpuppet version of that one as well?19:16
*** gyee_ has joined #openstack-infra19:16
cmurphydirk: not yet afaik19:17
*** rcernin has joined #openstack-infra19:20
*** jamesdenton has joined #openstack-infra19:20
*** gyee_ has quit IRC19:21
*** gyee has quit IRC19:22
*** markvoelker has quit IRC19:22
openstackgerritFélix Bouliane proposed openstack-infra/project-config master: Add ironic-pxe-snmp gate to virtualpdu  https://review.openstack.org/45694719:23
openstackgerritFélix Bouliane proposed openstack-infra/project-config master: Add ironic-pxe-snmp gate to virtualpdu  https://review.openstack.org/45694719:25
*** markvoelker has joined #openstack-infra19:30
*** liusheng has quit IRC19:30
*** chenying has quit IRC19:31
*** armax_ has joined #openstack-infra19:32
*** prateek has joined #openstack-infra19:33
*** signed8b_ has quit IRC19:33
*** armax has quit IRC19:33
*** armax_ is now known as armax19:33
*** tonytan_brb has quit IRC19:35
pabelangerhttps://review.openstack.org/#/c/147643/19:36
pabelangerlikely still need for long lived, but zuul workers we can remove once we land https://review.openstack.org/#/c/450029/19:36
dirkpabelanger: again, uid ranges is not the only change.. the other change is also needed19:37
pabelangerdirk: can you post a diff from the original19:38
dirkpabelanger: http://paste.openstack.org/show/606575/19:39
pabelangerwhy USER_GROUPS_ENAB19:40
*** rhallisey has quit IRC19:41
*** ssbarnea has quit IRC19:41
dirkpabelanger: because there is various code assuming that users have separate userspecific groups19:42
dirkwhich is not the case by default on suse, all users have "users" as initial group19:43
pabelangerright, in this case it would make more sense for the jobs to account for that, wouldn't it?19:43
pabelangeras an example, it is enabled right now for fedora-2519:44
pabelangerI'm trying to check centos-7 and xenial19:44
dirkpabelanger: its disabled everywhere..19:45
pabelangernot sure I follow, disabled where?19:46
dirkUSER_GROUPS_ENAB defaults to no if not given, and no other distro sets it afaik19:46
dirkpabelanger: ok, so I'll post a patch that removes login.defs everywhere?19:46
dirkand fix the fallout19:47
pabelangerdirk: once we land https://review.openstack.org/#/c/450029/ we likely can19:47
pabelangerI am hoping next week we can have nodepool use key-name setting and add our keys into root user via glean19:47
pabelangerclarkb:        server afs01.dfw.openstack.org partition /vicepa RW Site19:51
pabelangerfor docs volume19:51
clarkbcool so we are all done now (I guess get docs changes in?)19:52
pabelangeryup19:52
pabelangerstill dealing with mirror.npm19:52
*** chenying has joined #openstack-infra19:53
*** liusheng has joined #openstack-infra19:53
*** mortenie has joined #openstack-infra19:56
*** mhickey has quit IRC20:00
*** beslemon has joined #openstack-infra20:03
*** bhavik1 has joined #openstack-infra20:03
*** matrohon has joined #openstack-infra20:03
*** gyee has joined #openstack-infra20:08
*** yamamoto has joined #openstack-infra20:08
*** annegentle has quit IRC20:11
*** yamamoto has quit IRC20:13
*** Hal has joined #openstack-infra20:16
*** jascott1 has joined #openstack-infra20:18
*** chlong has joined #openstack-infra20:20
*** beslemon has quit IRC20:22
*** erikmwilson has joined #openstack-infra20:32
*** gouthamr has quit IRC20:35
*** jascott1 has quit IRC20:38
clarkbpabelanger: is it just behind?20:43
pabelangerclarkb: yes20:44
pabelangeronce vos release is finished, I'll mirror by hand and release again20:44
*** e0ne has joined #openstack-infra20:46
clarkbok20:46
*** ijw has joined #openstack-infra20:50
*** ijw has quit IRC20:50
*** fguillot has quit IRC20:50
*** bhavik1 has quit IRC20:50
*** ijw has joined #openstack-infra20:50
*** dimtruck is now known as zz_dimtruck20:52
*** esberglu has quit IRC20:54
*** chlong has quit IRC20:58
*** gouthamr has joined #openstack-infra20:58
*** anton_chev has quit IRC21:01
*** linzi has joined #openstack-infra21:01
*** pramodrj07 has joined #openstack-infra21:03
*** matrohon has quit IRC21:05
*** MasterOfBugs has quit IRC21:06
*** jkilpatr has joined #openstack-infra21:08
*** jascott1 has joined #openstack-infra21:10
*** e0ne has quit IRC21:10
*** jascott1 has quit IRC21:12
*** kzaitsev1mb has quit IRC21:14
*** claudiub|2 has joined #openstack-infra21:15
*** tonytan4ever has joined #openstack-infra21:17
*** slaweq has joined #openstack-infra21:18
slaweqmordred: hello, how are You?21:18
slaweqmordred: I just read Your email about shade and contributions to it21:18
*** prateek has quit IRC21:18
slaweqmordred: is it possible to help e.g. on transition to replace neutron client with REST calls?21:19
slaweqor someone is already doing it maybe?21:19
*** flepied has joined #openstack-infra21:24
*** mortenie has quit IRC21:28
*** jamesdenton has quit IRC21:31
*** mortenie has joined #openstack-infra21:39
*** felipemonteiro has quit IRC21:49
clarkbslaweq: hey, I don't know the answers to your questsions, but people do hang out in #openstack-shade and do shade specific things tehre that may not be here21:51
clarkbslaweq: its possible they (along with mordred) could provide answers to your questsions21:51
slaweqclarkb: ok, thx, I will join to openstack-shade then21:53
*** mdrabe has quit IRC21:58
*** amotoki has quit IRC21:58
*** jogo has quit IRC21:59
*** jogo has joined #openstack-infra21:59
*** MasterOfBugs has joined #openstack-infra22:01
*** jascott1 has joined #openstack-infra22:01
*** gouthamr has quit IRC22:03
*** pramodrj07 has quit IRC22:04
*** hashar has quit IRC22:07
*** jascott1 has quit IRC22:07
*** jkilpatr has quit IRC22:13
*** jascott1 has joined #openstack-infra22:17
*** hongbin has joined #openstack-infra22:17
*** feeblepipes has quit IRC22:20
*** jkilpatr has joined #openstack-infra22:20
*** rderose has joined #openstack-infra22:23
*** sdague has quit IRC22:25
*** rderose has quit IRC22:25
*** zz_dimtruck is now known as dimtruck22:26
*** thorst has quit IRC22:30
*** rcernin has quit IRC22:37
*** zul has quit IRC22:38
*** xarses has quit IRC22:44
sdakenot a pressing emergency - but spending lots of gate bandwidth just to run one job - can someone add an additonal +2 to this review: https://review.openstack.org/#/c/456130/22:44
sdakethanks :)22:44
*** askb has joined #openstack-infra22:47
*** Julien-zte has joined #openstack-infra22:49
*** flepied has quit IRC22:53
*** rbrndt has quit IRC22:55
*** esberglu has joined #openstack-infra22:55
*** xyang1 has quit IRC22:57
*** esberglu has quit IRC22:58
*** esberglu has joined #openstack-infra22:58
*** mortenie has quit IRC22:59
*** beslemon has joined #openstack-infra23:00
*** Julien-zte has quit IRC23:01
*** Julien-zte has joined #openstack-infra23:01
*** Julien-zte has quit IRC23:03
*** Julien-zte has joined #openstack-infra23:04
*** beslemon has quit IRC23:05
*** esberglu has quit IRC23:07
pabelangerclarkb: npm is now using 1TB of data23:09
pabelangermirror.npm                1000000000 1000000356  100%<<       70%    <<WARNING23:09
*** yamamoto has joined #openstack-infra23:11
pabelangergoing to bump it by 200GB for now23:12
*** baoli has quit IRC23:13
*** yamamoto has quit IRC23:16
pabelangerhttps://registry.npmjs.org/23:16
pabelangerif that is kb, it is 2.3 TB now23:16
pabelanger"data_size":236467862823:17
pabelangerclarkb: maybe we should reconsider the npm mirror and use the reverse proxy cache23:17
*** gouthamr has joined #openstack-infra23:20
*** beslemon has joined #openstack-infra23:25
*** wolverin_ has joined #openstack-infra23:31
*** wolverineav has quit IRC23:31
*** xarses has joined #openstack-infra23:33
*** xarses has quit IRC23:33
*** xarses has joined #openstack-infra23:33
*** beslemon has quit IRC23:37
*** beslemon_ has joined #openstack-infra23:37
*** beslemon_ has quit IRC23:41
*** wolverin_ has quit IRC23:46
*** wolverineav has joined #openstack-infra23:47
*** Julien-zte has quit IRC23:49
*** tosky has quit IRC23:50
*** baoli has joined #openstack-infra23:50
*** wolverineav has quit IRC23:53
*** baoli has quit IRC23:55
*** baoli has joined #openstack-infra23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!