| *** kerwin_bai has quit IRC | 00:07 | |
| *** BjoernT has quit IRC | 00:11 | |
| *** abitha has quit IRC | 00:25 | |
| *** skamithi14 has joined #openstack-ansible | 00:31 | |
| *** antonym has quit IRC | 00:34 | |
| *** coolj has quit IRC | 00:34 | |
| *** skamithi13 has quit IRC | 00:34 | |
| *** coolj has joined #openstack-ansible | 00:34 | |
| *** antonym has joined #openstack-ansible | 01:01 | |
| *** sdake has joined #openstack-ansible | 01:03 | |
| *** kerwin_bai has joined #openstack-ansible | 01:08 | |
| *** kerwin_bai has quit IRC | 01:26 | |
| *** skamithi14 has quit IRC | 01:33 | |
| *** skamithi13 has joined #openstack-ansible | 01:33 | |
| *** k_stev has joined #openstack-ansible | 02:01 | |
| *** sdake has quit IRC | 02:06 | |
| *** k_stev has quit IRC | 02:18 | |
| *** skamithi13 has quit IRC | 02:59 | |
| *** fawadkhaliq has joined #openstack-ansible | 03:41 | |
| openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible: Allow protocol to be set per endpoint-type https://review.openstack.org/226621 | 03:45 |
|---|---|---|
| *** tlian has quit IRC | 04:13 | |
| *** mcarden has quit IRC | 04:15 | |
| *** elo has joined #openstack-ansible | 04:16 | |
| *** mc has joined #openstack-ansible | 04:17 | |
| *** mc is now known as mcarden | 04:17 | |
| *** sdake has joined #openstack-ansible | 04:54 | |
| *** fawadkhaliq has quit IRC | 05:10 | |
| *** finchd has quit IRC | 05:24 | |
| *** finchd has joined #openstack-ansible | 05:28 | |
| *** sdake has quit IRC | 05:40 | |
| *** kukacz has joined #openstack-ansible | 05:46 | |
| *** elo has quit IRC | 05:50 | |
| *** elo has joined #openstack-ansible | 06:14 | |
| *** elo has quit IRC | 06:25 | |
| *** markvoelker has quit IRC | 06:27 | |
| *** daneyon has quit IRC | 07:00 | |
| *** daneyon has joined #openstack-ansible | 07:01 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update the AIO build convenience script https://review.openstack.org/222831 | 07:09 |
| *** neilus has joined #openstack-ansible | 07:10 | |
| *** elo has joined #openstack-ansible | 07:13 | |
| *** neilus has quit IRC | 07:13 | |
| *** neilus has joined #openstack-ansible | 07:14 | |
| *** gparaskevas has joined #openstack-ansible | 07:20 | |
| *** markvoelker has joined #openstack-ansible | 07:28 | |
| *** markvoelker has quit IRC | 07:33 | |
| *** elo has quit IRC | 07:40 | |
| *** elo has joined #openstack-ansible | 07:40 | |
| *** elo has quit IRC | 07:59 | |
| *** elo has joined #openstack-ansible | 07:59 | |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: Create utility log dir and link when on metal https://review.openstack.org/227169 | 08:11 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: Create utility log dir and link when on metal https://review.openstack.org/227169 | 08:12 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: Create utility log dir and link when on metal https://review.openstack.org/227169 | 08:13 |
| *** elo has quit IRC | 08:14 | |
| *** kerwin_bai has joined #openstack-ansible | 08:15 | |
| *** shoutm has joined #openstack-ansible | 08:41 | |
| *** javeriak has joined #openstack-ansible | 08:56 | |
| *** javeriak has quit IRC | 09:00 | |
| *** daneyon has quit IRC | 09:06 | |
| *** daneyon has joined #openstack-ansible | 09:07 | |
| openstackgerrit | Merged openstack/openstack-ansible: Update the AIO build convenience script https://review.openstack.org/222831 | 09:17 |
| *** markvoelker has joined #openstack-ansible | 09:29 | |
| *** markvoelker has quit IRC | 09:34 | |
| fxpester | hi all, is there any `low resource` version of openstack-ansible ? for like 4Gb RAM | 09:45 |
| odyssey4me | mhayden that happens every time in hpcloud-b4 :( | 09:46 |
| odyssey4me | fxpester right now we have a 8gb tested setup, 4gb would require not deploying more things and perhaps changing up a few things | 09:47 |
| fxpester | odyssey4me: how can I find it ? can you share link | 09:48 |
| odyssey4me | fxpester https://github.com/openstack/openstack-ansible/blob/master/development-stack.rst is the tested 8gb setup | 09:49 |
| fxpester | odyssey4me: thank you, looks very solid and Diagram is just epic! | 09:53 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Cinder to Liberty RC1 https://review.openstack.org/227205 | 10:12 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Cinder debugging https://review.openstack.org/215040 | 10:16 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Cinder debugging https://review.openstack.org/215040 | 10:19 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging https://review.openstack.org/215040 | 10:29 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging 2 https://review.openstack.org/215040 | 10:31 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Cinder debugging https://review.openstack.org/215040 | 10:39 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add policy changes required for OSSA-2015-018 / CVE-2015-5240 https://review.openstack.org/226872 | 10:51 |
| *** skamithi13 has joined #openstack-ansible | 10:52 | |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Cinder debugging https://review.openstack.org/215040 | 10:53 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add policy changes required for OSSA-2015-018 / CVE-2015-5240 https://review.openstack.org/226874 | 10:54 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add policy changes required for OSSA-2015-018 / CVE-2015-5240 https://review.openstack.org/226874 | 10:55 |
| *** kerwin_bai has quit IRC | 10:55 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add policy changes required for OSSA-2015-018 / CVE-2015-5240 https://review.openstack.org/226872 | 10:56 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Cinder debugging 2 https://review.openstack.org/215040 | 10:56 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Cinder debugging 3 https://review.openstack.org/215040 | 10:58 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging 4 https://review.openstack.org/215040 | 10:59 |
| mattt | sorry for spam :( | 10:59 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging 5 https://review.openstack.org/215040 | 10:59 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging 6 https://review.openstack.org/215040 | 11:01 |
| *** kerwin_bai has joined #openstack-ansible | 11:07 | |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging https://review.openstack.org/215040 | 11:07 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging 2 https://review.openstack.org/215040 | 11:15 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging 3 https://review.openstack.org/215040 | 11:21 |
| odyssey4me | mattt would you mind switching the subject to something like '[WIP] hpcloud-b4 debugging' ? ;) | 11:24 |
| odyssey4me | it'll help cloudnull and mhayden know that you're on it :) | 11:25 |
| mattt | yep sure | 11:25 |
| odyssey4me | thanks :) | 11:25 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] hpcloud-b4 debugging https://review.openstack.org/215040 | 11:25 |
| mattt | not having much luck getting a b4 :P | 11:27 |
| odyssey4me | unfortunately it's the luck of the draw\ | 11:28 |
| odyssey4me | if clarkb, fungi or jhesketh come online then perhaps they can pull a string to direct a build to a target zone :/ | 11:29 |
| *** markvoelker has joined #openstack-ansible | 11:30 | |
| odyssey4me | otherwise for now the only real option is to keep trying :/ | 11:30 |
| *** markvoelker has quit IRC | 11:34 | |
| *** skamithi14 has joined #openstack-ansible | 11:51 | |
| *** skamithi13 has quit IRC | 11:54 | |
| *** kerwin_bai has quit IRC | 12:01 | |
| fxpester | guys, is ansible is official deployment solution for HP Helion ? may be any other openstack distros ? | 12:09 |
| tiagogomes | I am trying to extend OSAD creating a parallel directory as I explained here http://docs.openstack.org/developer/openstack-ansible/developer-docs/extending.html . I'd like to use the vars in playbooks/roles/os_nova/defaults/main.yml . What's the best way of doing that? | 12:16 |
| tiagogomes | Using include_vars perhaps? | 12:17 |
| *** woodard has joined #openstack-ansible | 12:19 | |
| mattt | tiagogomes: that would work, but i'm not sure it's the most ansible way of doing it | 12:26 |
| tiagogomes | yes, that's what I thought. I am new to Ansible :) | 12:28 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Gate debugging https://review.openstack.org/215040 | 12:28 |
| *** markvoelker has joined #openstack-ansible | 12:31 | |
| mattt | tiagogomes: ;) | 12:33 |
| *** kerwin_bai has joined #openstack-ansible | 12:35 | |
| gparaskevas | Hello everyone | 12:35 |
| mhayden | mattt: if you get that figured out, you get hugs from me | 12:36 |
| gparaskevas | anyone have any clue on that? -> http://paste.openstack.org/show/473859/ | 12:36 |
| mattt | mhayden: i think we got it | 12:36 |
| gparaskevas | aio installation, kilo | 12:36 |
| gparaskevas | when i replay setup-openstack.yml i get this... | 12:36 |
| mhayden | mattt: then add a hug to my tally for the next time i'm in the UK | 12:36 |
| mattt | mhayden: https://review.openstack.org/#/c/215040/19/scripts/gate-check-commit.sh | 12:36 |
| mattt | mhayden: testing it now, but getting a b4 is proving to be difficult | 12:37 |
| mhayden | mattt: hey i heard that icanhazip.com site is just for malware :P | 12:37 |
| mattt | lolz | 12:37 |
| mattt | gparaskevas: what locale is your system using ? | 12:38 |
| gparaskevas | let me see i think greek though | 12:45 |
| mhayden | mattt: ah, you think the default AIO network blocks were clashing with something HP was using? | 12:46 |
| mattt | mhayden: it looks like b4 specifically uses that range | 12:47 |
| mattt | mhayden: b1 uses 10.0.0.0/24, b2, 10.0.1.0/24, etc. | 12:47 |
| mattt | (i think) | 12:47 |
| *** mgariepy has joined #openstack-ansible | 12:48 | |
| *** skamithi14 has quit IRC | 12:51 | |
| *** skamithi13 has joined #openstack-ansible | 12:52 | |
| rromans | this command in the deathmetal migration section: mysql cinder -e 'select host from volumes where deleted = 0; | 12:53 |
| rromans | should that not have "cinder" in it? | 12:53 |
| rromans | sorry, wrong window... | 12:54 |
| cloudnull | Fxpester there are quite a few deployers using it. As for distros Rackspace Private cloud is using it, if you'd lump rax in the distro category. I'm not sure what's powering hphelion these days. | 12:59 |
| cloudnull | Morning all ;-) | 13:00 |
| cloudnull | Much debugging, such [WIP], wow! | 13:01 |
| cloudnull | Mattt if we think that there are IP clashes with HP cloud we can change the net address and dhcp range for the lxc hosts by setting a couple of vars | 13:04 |
| cloudnull | https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_hosts/defaults/main.yml | 13:04 |
| mattt | cloudnull: https://review.openstack.org/#/c/215040/19/scripts/gate-check-commit.sh | 13:04 |
| mattt | cloudnull: morning btw :) | 13:05 |
| cloudnull | Well aren't you just an over achiever =) | 13:05 |
| cloudnull | How's it BTW? | 13:05 |
| * cloudnull goes back to his osic whole. | 13:06 | |
| *** kerwin_bai has quit IRC | 13:10 | |
| openstackgerrit | git-harry proposed openstack/openstack-ansible: Use Cinder-specific lvm.conf with LVM backend https://review.openstack.org/227277 | 13:11 |
| cloudnull | Wow auto correct hates me. s/whole/hole/ | 13:12 |
| *** shoutm has quit IRC | 13:13 | |
| *** alejandrito has joined #openstack-ansible | 13:14 | |
| mattt | cloudnull: ;) | 13:14 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Keystone to Liberty RC1 https://review.openstack.org/226917 | 13:17 |
| *** pradk has quit IRC | 13:17 | |
| *** tlian has joined #openstack-ansible | 13:36 | |
| *** pradk has joined #openstack-ansible | 13:37 | |
| pellaeon | Hello, my ./neutron-ha-tool.py --l3-agent-check always fail with: | 13:42 |
| pellaeon | ERROR {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} | 13:42 |
| pellaeon | which seems to be caused by using keystone v3 API as v2 | 13:43 |
| pellaeon | in openrc OS_AUTH_URL=http://172.29.236.19:5000/v3 | 13:44 |
| pellaeon | use neutron-ha-tool.py with -d shows that it make requests to http://172.29.236.19:5000/v3/tokens | 13:45 |
| pellaeon | I just upgraded from older kilo branch to the latest kilo branch | 13:47 |
| pellaeon | before that I removed my dedicated network host and use infra1-3 instead | 13:48 |
| gparaskevas | mattt: it was the LC_ALL wasnt defined at all | 13:48 |
| gparaskevas | mattt: I addes the same value as the rest of the LC variables and its ok now | 13:48 |
| *** KLevenstein has joined #openstack-ansible | 13:49 | |
| pellaeon | neutron-ha-tool.py fails after I move network hosts to infra1~3, I think, though I didn't discover that at that time | 13:49 |
| *** jwagner_away is now known as jwagner | 13:49 | |
| svg | git-harry: ping | 13:50 |
| mattt | gparaskevas: nice! | 13:50 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Do not use default LXC network in gate https://review.openstack.org/215040 | 13:50 |
| git-harry | svg: pong | 13:53 |
| tiagogomes | OSAD installs some stuff for logging right? How can I view the logs? Is there a webinterface? | 13:56 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: [WIP] Do not use default LXC network in gate https://review.openstack.org/215040 | 13:57 |
| svg | git-harry: looking at your proposed fix in https://review.openstack.org/#/c/227277/ | 13:59 |
| svg | isn't the one from roles/openstack_hosts/templates/lvm.conf.j2 going to overwrite lvm.conf on the metal hosts also? | 14:00 |
| svg | (I'm probably confused how this works, but making sure) | 14:00 |
| *** spotz_zzz is now known as spotz | 14:00 | |
| *** cloudtrainme has joined #openstack-ansible | 14:01 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Keystone to Liberty RC1 https://review.openstack.org/226917 | 14:02 |
| git-harry | svg: the idea was that by setting the using an lvm.conf file in /etc/cinder it doesn't matter what other roles modify the lvm.conf file in /etc/lvm | 14:04 |
| mattt | tiagogomes: rpc-openstack installs logstash, kibana, elasticsearch, etc. | 14:05 |
| mattt | tiagogomes: https://github.com/rcbops/rpc-openstack/tree/master/rpcd/playbooks/roles | 14:05 |
| git-harry | svg: actually I may have done something stupid, I'll stick it in WIP while I double check everything | 14:07 |
| tiagogomes | ah ok, so that stuff is not included with OSAD | 14:08 |
| *** Mudpuppy has joined #openstack-ansible | 14:15 | |
| *** skamithi14 has joined #openstack-ansible | 14:15 | |
| *** shoutm has joined #openstack-ansible | 14:16 | |
| *** Mudpuppy has quit IRC | 14:16 | |
| *** Mudpuppy has joined #openstack-ansible | 14:17 | |
| svg | git-harry: (was afk for a bit) thanks! | 14:17 |
| *** skamithi13 has quit IRC | 14:18 | |
| cloudnull | tiagogomes: OSAD has log shipping. if you want to see all of the logs your logging host has rsyslog recieving logs from all services. | 14:20 |
| tiagogomes | cloudnull yes I found it, thanks. Btw is there a file that containms all the logs for all services? | 14:21 |
| cloudnull | its a bind mount @ /openstack/<rsyslog_container_name>/log_store | 14:21 |
| cloudnull | then its broken out into the various containers as directories to keep all of the logs seperate . | 14:22 |
| cloudnull | if your wanting to stream the logs live you can: tail -f /openstack/<rsyslog_container_name>/log_store/**/*.log | 14:22 |
| tiagogomes | cloudnull ha, nice hack, thanks | 14:24 |
| cloudnull | also worth noting that the log setup ships to multiple hosts, if you have multiple logging hosts that is. and can be used to ship directly to things like splunk or loggly, how to do that is covered here : https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/rsyslog_client/defaults/main.yml#L51-L68 | 14:27 |
| cloudnull | that is if you have those services, and or you dont want to run your own elk stack as provided by rpc-openstack | 14:27 |
| *** fawadkhaliq has joined #openstack-ansible | 14:42 | |
| *** Bjoern_ has joined #openstack-ansible | 14:46 | |
| *** Bjoern_ is now known as BjoernT | 14:46 | |
| *** skamithi14 has quit IRC | 14:54 | |
| *** skamithi13 has joined #openstack-ansible | 14:55 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible-specs: Add spec for Gate Split https://review.openstack.org/221009 | 14:56 |
| *** neilus has quit IRC | 14:59 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update the AIO build convenience script https://review.openstack.org/227354 | 15:01 |
| *** shoutm has quit IRC | 15:05 | |
| *** skamithi14 has joined #openstack-ansible | 15:06 | |
| *** skamithi13 has quit IRC | 15:09 | |
| *** phalmos has joined #openstack-ansible | 15:10 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Cinder to Liberty RC1 https://review.openstack.org/227205 | 15:18 |
| *** kukacz has quit IRC | 15:23 | |
| mattt | odyssey4me: tried about 10x to get hpcloud-b4 to test https://review.openstack.org/#/c/215040/ but couldn't, i've let it run through to validate it hasn't broken anything in general | 15:26 |
| mattt | odyssey4me: should i remove the WIP and we just merge it, and remove it if it doesn't solve the -b4 issues? | 15:26 |
| odyssey4me | mattt our best chance to see is by merging it | 15:27 |
| mattt | odyssey4me: alright | 15:27 |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: Do not use default LXC network in gate https://review.openstack.org/215040 | 15:28 |
| *** shoutm has joined #openstack-ansible | 15:28 | |
| odyssey4me | mattt well spotted for finding the most likely cause for the gate plague :) | 15:38 |
| mattt | amazing how quick problems are to solve when you can actually log into the instance :P | 15:43 |
| evrardjp | hello everyoe | 15:51 |
| evrardjp | everyone* | 15:53 |
| mancdaz | odyssey4me cloudnull so the 'not arp cache flush' patch is taking longer because whenever we are waiting for container ssh, it takes an age | 15:55 |
| mancdaz | manually issuing an arp -d for the particular container allows the run to continue | 15:56 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-specs: Spec: Security Hardening https://review.openstack.org/222619 | 15:56 |
| *** elo has joined #openstack-ansible | 15:56 | |
| *** skamithi14 has quit IRC | 15:56 | |
| *** skamithi13 has joined #openstack-ansible | 15:57 | |
| evrardjp | mancdaz: -d? | 15:57 |
| mancdaz | odyssey4me cloudnull seems the mac address of the container interface is changing | 15:57 |
| evrardjp | oh that explains that | 15:57 |
| mancdaz | evrardjp delete the entry | 15:58 |
| evrardjp | yeah I know ;) | 15:58 |
| *** galstrom_zzz is now known as galstrom | 16:00 | |
| odyssey4me | mancdaz ah, so the container restart results in the new veth and thus a mac change | 16:01 |
| mancdaz | odyssey4me right | 16:01 |
| *** fawadkhaliq has quit IRC | 16:01 | |
| evrardjp | odyssey4me: isn't it better to generate a mac-address for the containers? | 16:02 |
| mancdaz | urgh I commented in the review but formatting fail | 16:02 |
| *** fawadkhaliq has joined #openstack-ansible | 16:02 | |
| evrardjp | lxc config can set the mac address IIRC | 16:02 |
| mancdaz | evrardjp something that doesn't change? | 16:02 |
| evrardjp | yeah like that | 16:02 |
| evrardjp | when the inventory is created, we could define a mac address for the containers (when is_metal is false) | 16:03 |
| evrardjp | each container would have its mac address used in lxc config | 16:03 |
| evrardjp | and it wouldn't move | 16:03 |
| *** phalmos has quit IRC | 16:04 | |
| Apsu | Don't use arp -d, use ip n(eighbor), btw. | 16:04 |
| mancdaz | Apsu heh I was waiting for you to tell me that :) | 16:04 |
| Apsu | evrardjp: Statically defining MACs is hard, just like IPs you need a pool and have to keep track so you don't assign dupes. | 16:04 |
| evrardjp | arp -d works on almost all unixes ;) | 16:04 |
| evrardjp | Apsu: we already have an inventory | 16:05 |
| mancdaz | aside from deprecation, any difference in behaviour? | 16:05 |
| Apsu | evrardjp: Yes, but it's old, unmaintained and uses old and incomplete kernel interfaces. | 16:05 |
| Apsu | Like the rest of net-tools | 16:05 |
| *** fawadkhaliq has quit IRC | 16:05 | |
| evrardjp | I don't think so mancdaz, it's jsut the future | 16:05 |
| evrardjp | Apsu: true | 16:05 |
| Apsu | mancdaz: I don't know of a specific deficiency with arp (whereas I know of a ton with ifconfig/netstat/route), but... | 16:06 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Cinder to Liberty RC1 https://review.openstack.org/227205 | 16:06 |
| Apsu | Old and unmaintained, lots of open bugs. Same as all of net-tools :) | 16:06 |
| mancdaz | Apsu well arp lets me specify a container, ip neig del wants an ip address | 16:06 |
| mancdaz | and I'm lazy | 16:06 |
| Apsu | mancdaz: It doesn't let you specify a container, it lets you specify a hostname that it will attempt to resolve to an IP. Which won't work for what you want because we need to clear all IPs in all CIDRs the container was using :) | 16:07 |
| mancdaz | yeah that's what I said | 16:07 |
| Apsu | lol | 16:08 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-specs: Spec: Security Hardening https://review.openstack.org/222619 | 16:08 |
| Apsu | Note the "won't work" part ;P | 16:08 |
| mancdaz | in my tiny use case vacuum, it works fine cuz each container only has a single entry in the arp table | 16:08 |
| Apsu | lol | 16:08 |
| Apsu | Your managerial technical domain is showing | 16:08 |
| mancdaz | Apsu yw? | 16:08 |
| Apsu | hehe | 16:08 |
| mancdaz | I've learnt that management is not about being smart, just surrounding yourself with smart people | 16:09 |
| evrardjp | :) | 16:09 |
| odyssey4me | mancdaz surely we can implement a task to simply remove the arp entry for that ip? | 16:09 |
| mancdaz | so if I fail, it's your fault | 16:09 |
| *** alop has joined #openstack-ansible | 16:09 | |
| Apsu | I think there's also a component of listening to those smart people. Not sure on that part. | 16:09 |
| Apsu | Seems to not be the case in practice | 16:09 |
| evrardjp | something interesting from the lxc doc on linuxcontainers.org | 16:09 |
| mancdaz | Apsu say that again? I didn't hear | 16:09 |
| evrardjp | "lxc.network.hwaddr" | 16:09 |
| Apsu | odyssey4me: Yep. That was my suggestion which cloudnull implemented I believe. | 16:09 |
| mancdaz | that does not work so well in irc | 16:10 |
| evrardjp | apsu: check that part about the conflicts ;) | 16:10 |
| Apsu | evrardjp: Yeah that's the static/dynamic MAC bit. We currently use a template. | 16:10 |
| evrardjp | what I meant is, with how the doc is written, it looks like the mac is generated when you don't give a value, but there isn't any check to see if the mac is already in use | 16:11 |
| Apsu | Meaning a MAC template, with the static parts set and the dynamic parts as xx:xx:xx | 16:11 |
| *** KLevenstein has quit IRC | 16:11 | |
| Apsu | I know the doc doesn't say it, but I haven't yet seen a conflict in thousands of containers being built. I suspect it checks the neighbor table. | 16:11 |
| Apsu | Or sends an ARP request | 16:11 |
| *** KLevenstein has joined #openstack-ansible | 16:11 | |
| mancdaz | Apsu what is the timeout on stale entries? | 16:12 |
| odyssey4me | ah, it was only done on container destroy: https://review.openstack.org/#/c/225367/3/playbooks/lxc-containers-destroy.yml,cm | 16:12 |
| Apsu | odyssey4me: Yeah, that's the only place it's needed at all. | 16:12 |
| Apsu | Never on install/restart | 16:12 |
| odyssey4me | whereas, if I understand it correctly, this should be done on container restart due to the mac change? | 16:13 |
| *** phalmos has joined #openstack-ansible | 16:13 | |
| Apsu | Hrm. You're saying for a given container that doesn't have a config change, it doesn't cache the MAC it generated, and makes a new one each start? | 16:13 |
| evrardjp | It's weird that this happens on restart | 16:13 |
| Apsu | I didn't think that was the case | 16:14 |
| mancdaz | Apsu that's what I'm seeing | 16:14 |
| *** woodard has quit IRC | 16:14 | |
| Apsu | mancdaz: Interesting. | 16:14 |
| Apsu | So, I had proposed an alternative mechanism for dealing with this automatically, in my bug report | 16:15 |
| odyssey4me | so it seems to me that we should be registering the result of the tasks that cause a container restart, then deleting the arp entry if those tasks have the 'changed' status | 16:15 |
| Apsu | We can set arp_notify to 1 for container interfaces | 16:16 |
| mancdaz | Apsu https://gist.github.com/mancdaz/4f1e5c060731e384f482 | 16:16 |
| Apsu | And cycle them so they send out gratuitous ARPs | 16:16 |
| Apsu | Then we don't need to do any ARP management on hosts at all | 16:16 |
| odyssey4me | Apsu sounds good, but will that require a container restart again? | 16:16 |
| *** woodard has joined #openstack-ansible | 16:16 | |
| mancdaz | Apsu that sounds like the win | 16:16 |
| Apsu | odyssey4me: Nope. | 16:16 |
| odyssey4me | that sounds like a win then | 16:17 |
| Apsu | 1 - Generate gratuitous arp requests when device is brought up | 16:17 |
| Apsu | or hardware address changes. | 16:17 |
| mancdaz | Apsu how does one implement that | 16:17 |
| Apsu | So, we may need a little bit of post-up stuff in the /etc/network/interface file(s) in the container | 16:17 |
| Apsu | Because I think ifupdown (the Ubuntu network scripts that deal with interface configs) bring interfaces up Then add IPs. | 16:18 |
| Apsu | And that won't work. | 16:18 |
| Apsu | It has to have the IP and then be brought up | 16:18 |
| Apsu | So... we can add some post-up lines that ip link set down, ip link set up | 16:18 |
| Apsu | That should do the needful | 16:18 |
| Apsu | If you've got a box I can hop on I can test it | 16:18 |
| Apsu | I did an arp_notify solution in the VIP namespace magic I made for RPC v4 | 16:19 |
| *** markvoelker_ has joined #openstack-ansible | 16:21 | |
| evrardjp | let's hope nobody want to use ipv6 | 16:24 |
| evrardjp | on these interfaces | 16:24 |
| evrardjp | I don't know how ND works (and its timers) | 16:24 |
| evrardjp | ND/NA | 16:24 |
| *** markvoelker has quit IRC | 16:24 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Keystone to Liberty RC1 https://review.openstack.org/226917 | 16:24 |
| *** cloudtrainme has quit IRC | 16:24 | |
| *** jwagner has quit IRC | 16:24 | |
| Apsu | https://github.com/rcbops-cookbooks/keepalived/blob/1c3d04b543a70dae3007a653718e5d9067b279a3/files/default/notify.sh#L29 | 16:24 |
| Apsu | And | 16:24 |
| Apsu | https://github.com/rcbops-cookbooks/keepalived/blob/1c3d04b543a70dae3007a653718e5d9067b279a3/files/default/notify.sh#L46-L48 | 16:24 |
| Apsu | Set notify, cycle interface. | 16:24 |
| Apsu | Now... if there are default or other custom routes, we'll have to re-add those. | 16:24 |
| Apsu | This is all assuming that ifupdown doesn't do the needful. | 16:24 |
| *** cloudtra_ has joined #openstack-ansible | 16:24 | |
| Apsu | But all of that can be templated into place relatively easily | 16:24 |
| *** jwagner- has joined #openstack-ansible | 16:24 | |
| *** afranc202 has quit IRC | 16:24 | |
| *** jwagner- is now known as jwagner | 16:24 | |
| Apsu | evrardjp: It's actually the same, essentially, and has similar sysctls | 16:24 |
| *** iraw- has joined #openstack-ansible | 16:24 | |
| evrardjp | ok | 16:24 |
| evrardjp | I'm more concerned about the mac change | 16:24 |
| Apsu | ndisc_notify: | 16:25 |
| evrardjp | ok | 16:25 |
| evrardjp | :) | 16:25 |
| Apsu | :D | 16:25 |
| evrardjp | the mac change shouldn't happen at all... sending gratuitous arp isn't really bad per se, but I'd rather avoid these | 16:25 |
| Apsu | If we start assigning v6 IPs, we won't be using SLAAC I imagine | 16:25 |
| evrardjp | mmm: I'd rather avoid the mac address change* | 16:25 |
| Apsu | why would you rather avoid them? They're exactly what we want to accomplish -- updating all hosts to know the new MAC | 16:25 |
| Apsu | You'd rather have a bunch of Ansible hooks to manually fudge it? | 16:26 |
| evrardjp | no, it's good that way | 16:26 |
| mancdaz | Apsu this is a sysctl thing, so we can just throw it in all containers? | 16:26 |
| evrardjp | I'd rather have my mac address table stable | 16:26 |
| Apsu | evrardjp: Ah. Well that's a larger project, but certainly doable. | 16:26 |
| odyssey4me | yeah, can't we just do 'sysctl -w net.ipv4.conf.all.arp_notify=1' in all containers? | 16:27 |
| evrardjp | In my past I've seen some switches that didn't like the mac address changes at all | 16:27 |
| evrardjp | because it's handled by cpu instead of the ASIC (IIRC) | 16:27 |
| Apsu | odyssey4me: mancdaz: Yes, but, sysctls are shared. Specific interface entries will only be visible in their containers, but shared ones like conf.all will apply everywhere, including the host. | 16:27 |
| Apsu | Since interfaces will come up/down a lot potentially with various Ansible/VM tasks, I'd rather not do that | 16:28 |
| Apsu | Instead, set it to on for each container interface. | 16:28 |
| odyssey4me | Apsu ok, and do we do that on the host or inside the container? | 16:28 |
| evrardjp | what I don't get: I've restarted a container here, and it didn't change mac address | 16:28 |
| mancdaz | Apsu it seems like the only interfaces that are going to come up/down a lot are the container interfaces anyway, so what difference? | 16:28 |
| odyssey4me | mancdaz consider the neutron agent container - a whole different story there | 16:29 |
| odyssey4me | also all compute hosts have vm's spinning up and down all the time | 16:29 |
| Apsu | mancdaz: It's not a huge deal either way, honestly. GratARPs are small, and ARP is going to happen a lot whether we do it or not. | 16:29 |
| mancdaz | Apsu right | 16:30 |
| evrardjp | yup | 16:30 |
| mancdaz | seems like an easier fix to use the sysctl module | 16:30 |
| Apsu | mancdaz: So, if you set arp_notify (make sure you use sysctl.conf so it applies on boot) on the mgmt container interface, and ifdown/ifup, does it gratarp? | 16:30 |
| Apsu | Since you have a container you're testign on | 16:31 |
| Apsu | And/or does it gratarp on lxc-stop/lxc-start | 16:31 |
| Apsu | FYI, a gratarp is just an ARP reply with a particular configuration of source/dest components. | 16:33 |
| mancdaz | Apsu I need to scoot, but I pinged you the instance details I'm testing on | 16:33 |
| Apsu | It's not a special piece of traffic per se | 16:33 |
| Apsu | mancdaz: kk | 16:33 |
| mancdaz | Apsu make the magics | 16:33 |
| mancdaz | fix the fixes https://review.openstack.org/#/c/225367/ | 16:34 |
| Apsu | That's what I do | 16:34 |
| Apsu | magic++ | 16:34 |
| *** alop has quit IRC | 16:43 | |
| *** elo has quit IRC | 16:47 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Cinder to Liberty RC1 https://review.openstack.org/227205 | 16:50 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Keystone to Liberty RC1 https://review.openstack.org/226917 | 16:51 |
| *** woodard_ has joined #openstack-ansible | 16:52 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Do not use default LXC network in gate https://review.openstack.org/215040 | 16:52 |
| *** woodard has quit IRC | 16:55 | |
| *** pradk has quit IRC | 16:57 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add policy changes required for OSSA-2015-018 / CVE-2015-5240 https://review.openstack.org/226874 | 16:58 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Fix for keystone LDAP pkg missing https://review.openstack.org/226740 | 16:59 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Allow tempest to deploy when no heat in environment https://review.openstack.org/226727 | 17:00 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update the AIO build convenience script https://review.openstack.org/227354 | 17:01 |
| *** shoutm has quit IRC | 17:11 | |
| *** elo has joined #openstack-ansible | 17:15 | |
| *** woodard_ has quit IRC | 17:17 | |
| *** jwagner is now known as jwagner_away | 17:20 | |
| Apsu | Got it. | 17:24 |
| Apsu | Veth pairs behave differently on link up/down since they've got two ends hooked together, and don't send gratarps when either end or both are cycled. | 17:25 |
| Apsu | But! | 17:25 |
| Apsu | Setting the MAC works, including when you set it to the same value it already has. | 17:25 |
| Apsu | So all we need is a post-up with: ip link set $IFACE address $(</sys/class/net/$IFACE/address) | 17:25 |
| Apsu | And arp_notify on | 17:25 |
| Apsu | It can be set as net.ipv4.conf.all.arp_notify=1 | 17:26 |
| Apsu | With the default arp_accept=0 on the host, it won't add entries for IPs that haven't been contacted yet, but it will update existing entries. | 17:26 |
| Apsu | We can also set arp_accept to 1 if we want, so we won't have to ARP for containers the first time we contact them after they come up. It's a very minor time savings, but it's an option. | 17:27 |
| Apsu | @ mancdaz evrardjp odyssey4me cloudnull | 17:27 |
| *** gparaskevas has quit IRC | 17:40 | |
| mhayden | for anyone following the security-hardening spec, it's been adjusted to follow DISA STIGs | 17:45 |
| *** phalmos has quit IRC | 17:50 | |
| palendae | mhayden: I knew those words up to DISA | 18:03 |
| mhayden | palendae: http://docs-draft.openstack.org/19/222619/7/check/gate-openstack-ansible-specs-docs/2867df1//doc/build/html/specs/mitaka/security-hardening.html | 18:06 |
| stevelle | enqueued | 18:20 |
| *** woodard has joined #openstack-ansible | 18:24 | |
| openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible: Install spice-html5 from source https://review.openstack.org/226462 | 18:26 |
| *** phalmos has joined #openstack-ansible | 18:29 | |
| *** woodard_ has joined #openstack-ansible | 18:30 | |
| *** woodard has quit IRC | 18:32 | |
| *** elo has quit IRC | 18:37 | |
| *** alop has joined #openstack-ansible | 18:51 | |
| *** woodard has joined #openstack-ansible | 18:52 | |
| *** jwagner_away is now known as jwagner | 18:52 | |
| *** woodard_ has quit IRC | 18:55 | |
| *** abitha has joined #openstack-ansible | 18:56 | |
| evrardjp | Apsu: nice trick the ip link set :) | 19:29 |
| Apsu | evrardjp: That's what they pay me for. I have the biggest bag of tricks ;P | 19:30 |
| evrardjp | :) | 19:32 |
| evrardjp | for the arp_accept to 1, it's really minor, right? | 19:33 |
| Apsu | Yeah, the difference between not sending an ARP/getting a reply vs sending/receiving one | 19:33 |
| evrardjp | becase on first time we contact them we just do the arp as usual | 19:33 |
| Apsu | Which is usually a few ms | 19:34 |
| evrardjp | ok I understood correctly \o/ | 19:34 |
| Apsu | lol yep | 19:34 |
| evrardjp | with 0, the update is fine then? | 19:34 |
| evrardjp | (of the existing ones) | 19:34 |
| Apsu | Yes | 19:34 |
| evrardjp | it would be bad to encounter another bug right there ;) | 19:35 |
| Apsu | Existing entries always get updated by a gratarp regardless of arp_accept | 19:35 |
| evrardjp | ofc | 19:35 |
| Apsu | That's what the docs say and what I observed :) | 19:35 |
| evrardjp | I still wonder why the mac changes | 19:35 |
| palendae | lxc destroys the veth when it goes down, I think | 19:36 |
| palendae | And doesn't record it | 19:36 |
| palendae | So it regenerates based on the template when it re-ups | 19:36 |
| palendae | If I understand correctly | 19:36 |
| evrardjp | that's what weird: I tried at home, and it took the same mac | 19:36 |
| evrardjp | I'll retry | 19:36 |
| palendae | Huh | 19:36 |
| palendae | I'm kind of speculating | 19:37 |
| evrardjp | I'll double check if I'm well on veth | 19:37 |
| Apsu | It could cache or not. I assumed it did, mancdaz said it didn't and illustrated evidence. | 19:37 |
| Apsu | It might vary by version, distro packaging, default config flags, etc. Hard to say | 19:37 |
| evrardjp | (it's a day off today, I don't have access to my tst or prod clouds, so it's test with what I have home :p) | 19:37 |
| Apsu | Regardless, MAC changes or not, arp_notify covers everything | 19:37 |
| evrardjp | true | 19:38 |
| palendae | evrardjp: Sure, and you're testing lxc, which is installable on any linux box | 19:38 |
| evrardjp | Apsu: I'm not familiar with the arp_notify: it sends gratarp only when you're using your link set, or there are more events that trigger this? | 19:40 |
| evrardjp | (I mean other than ip link * events) | 19:41 |
| Apsu | evrardjp: link up or MAC set | 19:41 |
| Apsu | That's it | 19:41 |
| evrardjp | ok | 19:41 |
| Apsu | Same with ndisc_notify for v6 | 19:41 |
| *** alejandrito has quit IRC | 19:53 | |
| *** woodard_ has joined #openstack-ansible | 19:53 | |
| *** woodard has quit IRC | 19:56 | |
| *** kukacz has joined #openstack-ansible | 19:58 | |
| *** elo has joined #openstack-ansible | 20:07 | |
| *** woodard has joined #openstack-ansible | 20:11 | |
| *** woodard_ has quit IRC | 20:12 | |
| *** elo has quit IRC | 20:12 | |
| *** woodard_ has joined #openstack-ansible | 20:13 | |
| *** woodard has quit IRC | 20:16 | |
| *** elo has joined #openstack-ansible | 20:21 | |
| *** k_stev has joined #openstack-ansible | 20:23 | |
| *** elo has quit IRC | 20:25 | |
| *** elo has joined #openstack-ansible | 20:28 | |
| *** metral is now known as metral_zzz | 20:36 | |
| *** metral_zzz is now known as metral | 20:36 | |
| *** KLevenstein has quit IRC | 20:43 | |
| openstackgerrit | Merged openstack/openstack-ansible: Put horizon in its own process https://review.openstack.org/226889 | 20:53 |
| *** elo has quit IRC | 20:59 | |
| *** daneyon has quit IRC | 21:04 | |
| *** daneyon has joined #openstack-ansible | 21:05 | |
| openstackgerrit | Merged openstack/openstack-ansible: Update juno SHA's - 23 Sep 2015 https://review.openstack.org/226861 | 21:07 |
| *** galstrom is now known as galstrom_zzz | 21:10 | |
| *** woodard_ has quit IRC | 21:10 | |
| *** phalmos has quit IRC | 21:14 | |
| *** Mudpuppy_ has joined #openstack-ansible | 21:32 | |
| *** kukacz has quit IRC | 21:33 | |
| *** Mudpuppy has quit IRC | 21:36 | |
| *** Mudpuppy_ has quit IRC | 21:37 | |
| *** elo has joined #openstack-ansible | 21:41 | |
| *** iraw- has quit IRC | 21:41 | |
| *** alejandrito has joined #openstack-ansible | 21:41 | |
| *** iraw- has joined #openstack-ansible | 21:42 | |
| *** daneyon has quit IRC | 21:49 | |
| *** daneyon has joined #openstack-ansible | 21:50 | |
| *** jlvillal has quit IRC | 21:51 | |
| openstackgerrit | Miguel Grinberg proposed openstack/openstack-ansible: Put horizon in its own process https://review.openstack.org/227570 | 21:53 |
| *** jlvillal has joined #openstack-ansible | 21:57 | |
| *** jwagner is now known as jwagner_away | 21:57 | |
| openstackgerrit | Merged openstack/openstack-ansible: Update kilo SHA's - 23 Sep 2015 https://review.openstack.org/226890 | 22:15 |
| *** jhesketh has quit IRC | 22:28 | |
| *** jhesketh has joined #openstack-ansible | 22:29 | |
| *** spotz is now known as spotz_zzz | 22:33 | |
| openstackgerrit | Merged openstack/openstack-ansible: Allow tempest to deploy when no heat in environment https://review.openstack.org/226727 | 22:52 |
| *** k_stev has quit IRC | 22:56 | |
| *** alejandrito has quit IRC | 23:03 | |
| *** markvoelker_ has quit IRC | 23:04 | |
| openstackgerrit | Steve Lewis proposed openstack/openstack-ansible: Put horizon in its own process https://review.openstack.org/227570 | 23:11 |
| *** cloudtra_ has quit IRC | 23:18 | |
| *** cloudtrainme has joined #openstack-ansible | 23:18 | |
| *** cloudtrainme has quit IRC | 23:23 | |
| *** skamithi13 has quit IRC | 23:24 | |
| *** skamithi13 has joined #openstack-ansible | 23:24 | |
| *** agireud has quit IRC | 23:42 | |
| *** shoutm has joined #openstack-ansible | 23:45 | |
| *** harlowja has quit IRC | 23:56 | |
| *** abitha has quit IRC | 23:56 | |
| *** thrawn01 has quit IRC | 23:56 | |
| *** darrenc has quit IRC | 23:57 | |
| *** neillc has quit IRC | 23:57 | |
| *** gus has quit IRC | 23:57 | |
| *** bogeyon18 has quit IRC | 23:57 | |
| *** harlowja has joined #openstack-ansible | 23:57 | |
| *** darrenc_ has joined #openstack-ansible | 23:57 | |
| *** darrenc_ is now known as darrenc | 23:58 | |
| *** neillc has joined #openstack-ansible | 23:58 | |
| *** thrawn01 has joined #openstack-ansible | 23:58 | |
| *** bogeyon18 has joined #openstack-ansible | 23:58 | |
| *** gus has joined #openstack-ansible | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!