Monday, 2018-01-29

*** cjloader has joined #openstack-ansible00:02
xdfilglance container isn't mounting NFS at start from fstab. But if I do "mount -a" it mounts fine. it seems like the container config is simply ignoring fstab on boot00:07
*** lbragstad has joined #openstack-ansible00:14
*** cjloader has quit IRC00:40
*** cjloader has joined #openstack-ansible00:40
*** cjloader has quit IRC00:45
*** cjloader has joined #openstack-ansible00:56
*** cjloader has quit IRC01:01
*** rstarmer has quit IRC01:01
mnaserhmm01:03
mnaserosa doesn't do any active/passive setup for rbd, does it?01:04
mnaserit seems to deploy 3 different and independent hosts which means that while the cluster is still up, if a host goes down, all volumes linked to it wont be able to perform their work01:05
mnaservs active/passive where they all share the same host name and then when one goes down, any other one can go up and take over01:06
logan-mnaser: yeah you can do that01:57
logan-it is not in the docs afaik01:57
logan-you're talkign about setting the cinder-volume hostname right?01:57
mnaserlogan-: what coordinates the active/passive stuff though?  and yes01:57
mnasercause i think if you run the same host across all 3, only 1 instance should run at time01:58
logan-ive always run them active/active with the same hostname01:58
mnaseri guess i'll have to do some reading to check (though your feedback helps)01:59
mnaseri know tripleo uses pacemaker to do active/passive01:59
logan-i could see needing to do that if you actually have cinder-volume in the data path for iscsi or something01:59
logan-but for ceph it is basically just a metadata service01:59
mnaserlogan-: im thinking in some cases of potentially having races02:00
mnaserbut yeah i see your logic02:00
logan-im curious to hear what you find out. i was a concerned about the MQ side when I saw that advised somewhere in the juno/kilo timeframe, but ive run with it for that long and have not seen any problems02:02
mnaserlogan-: if anything this would help the scale side of things02:03
mnaseri'm seeing some really old discussions02:04
xdfilhi02:04
mnaserlogan-: ill check with the cinder team and see what they have to say about it02:11
logan-cool02:11
*** cjloader has joined #openstack-ansible02:15
*** cjloader has quit IRC02:40
*** hw_wutianwei_ has joined #openstack-ansible02:43
*** dave-mccowan has quit IRC03:20
*** gkadam_ has quit IRC03:58
*** pmannidi has quit IRC04:47
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_horizon master: Zuul: Remove project name  https://review.openstack.org/53874905:07
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_horizon stable/newton: Zuul: Remove project name  https://review.openstack.org/53875405:27
*** rstarmer has joined #openstack-ansible05:31
*** gkadam has joined #openstack-ansible05:47
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_horizon stable/ocata: Zuul: Remove project name  https://review.openstack.org/53875705:47
*** hybridpollo has quit IRC05:48
*** rstarmer has quit IRC05:50
*** ivveh has joined #openstack-ansible05:59
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_horizon stable/pike: Zuul: Remove project name  https://review.openstack.org/53876306:07
*** rstarmer has joined #openstack-ansible06:07
*** hybridpollo has joined #openstack-ansible06:08
*** ANKITA has joined #openstack-ansible06:14
*** hybridpollo has quit IRC06:27
openstackgerritTuan Luong-Anh proposed openstack/ansible-hardening master: Zuul: Remove project name  https://review.openstack.org/53877506:28
*** ANKITA has quit IRC06:35
*** yolanda_ has quit IRC06:44
evrardjphello everyone -- FYI I am freezing the a-r-r https://review.openstack.org/#/c/538680/ for master, to prepare for milestone 306:51
evrardjpxdfil: bugs can be reported on https://bugs.launchpad.net/openstack-ansible06:52
ivvehgood morning06:52
evrardjpxdfil: for mounting on boot, I am not sure which systemd service take care of that. But if there is a bug, please report it :)06:53
evrardjpivveh: good morning!06:53
*** ivveh has quit IRC07:14
*** rstarmer has quit IRC07:14
*** armaan has joined #openstack-ansible07:26
*** pcaruana has joined #openstack-ansible07:44
*** mbuil has joined #openstack-ansible08:00
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Update all SHAs for 17.0.0.0b3  https://review.openstack.org/53868008:03
evrardjpjmccrory: when you're available, I've added you to the review of https://review.openstack.org/#/c/538315/08:09
*** hamza21 has joined #openstack-ansible08:09
evrardjpthis should unblock O to P upgrades if I am not mistaken.08:09
* prometheanfire hates the integrated gate08:16
evrardjpwhy?08:17
evrardjpthey didn't do anything wrong :p08:17
evrardjpyes they are slow08:17
evrardjpbut not to the point I'd hate them !08:17
prometheanfireit keeps on randomly failing my patches08:18
prometheanfire(and I do mean random)08:18
prometheanfireI'm thinking an extra 5 min would 'fix' it08:18
evrardjpin which branch?08:18
prometheanfirewonder if it could be blamed on meltdown...08:18
evrardjpPike?08:18
prometheanfirenot-osa08:18
prometheanfirerequirements08:18
evrardjpoh ok08:18
evrardjpyou can change the timeout08:19
prometheanfirejust me venting is all08:19
evrardjphow long is your job rn?08:19
prometheanfireback up to 15 hours :|08:19
evrardjpwoa08:19
evrardjpoh it's not your job that last 15h08:19
evrardjpit's the integrated queue08:20
evrardjpof fun, you have post merge issues too.08:20
prometheanfiredoesn't look like we define it in our .zuul.d08:20
evrardjphttp://logs.openstack.org/c5/c5053646aa1bbbd0b2f2a5b269ddb42a9f29d49e/post/publish-openstack-python-branch-tarball/8205ccb/job-output.txt.gz08:21
prometheanfirethat did get to github at least...08:22
prometheanfirehttps://github.com/openstack/requirements/commit/c5053646aa1bbbd0b2f2a5b269ddb42a9f29d49e08:22
evrardjpthey all inherit from openstack-tox08:22
evrardjpbut you shouldn't probably change it, your job timeouts are fine I think08:22
evrardjpwell to what I see rn08:23
evrardjpit's just that the queue is big08:23
evrardjppost-merge08:23
evrardjpwhat I meant08:23
evrardjpit's on the tarballs I guess08:23
evrardjpthat's not recheckable08:24
evrardjpmaybe worth discussing it with infra08:24
evrardjpbut yeah, if you think you have jobs timeouts that are too painful, just change the parent, create a new parent for tox-requirements, and change the timeout there08:25
evrardjpjust need to add this parent into jobs.yaml if I am not mistaken, with just this override.08:25
evrardjpshould be a 3 line change, and a sed.08:25
evrardjpwell, more a sed and then a 3 line addition :)08:26
evrardjp4 line*08:26
prometheanfirestuff like https://review.openstack.org/53679308:26
evrardjpyu had many issues due to legacy-tempest-dsvm-neutron-full and legacy-requirements-integration-dsvm. on top of that, they take a lot of time. but yeah you're hitting more and more the 30+ minutes mark in your cross-jobs.08:28
evrardjpneutron and nova being the ones that take a little longer, why not only changing those 2?08:28
evrardjplet me check the stats quick08:29
evrardjpyeah cross-neutron-py35 seem to be a more regular offender, but it's not that bad08:31
evrardjplook at http://zuul.openstack.org/builds.html , pipeline gate (or check), job: cross-neutron-py35, project: openstack/requirements08:32
evrardjpit's not too bad :D08:32
*** peri has joined #openstack-ansible08:36
*** sxc731 has joined #openstack-ansible08:41
*** masber has quit IRC08:41
*** hamza21 has quit IRC08:42
*** masber has joined #openstack-ansible08:42
*** hamza21 has joined #openstack-ansible08:42
*** cjloader has joined #openstack-ansible08:48
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249108:51
*** cjloader has quit IRC08:53
*** cxc99 has quit IRC08:57
*** cxc99 has joined #openstack-ansible09:00
*** jafeha has joined #openstack-ansible09:17
jafehagood morning09:19
prometheanfiremoin09:19
hwoaranggood morning09:20
evrardjpgood morning jafeha hwoarang09:28
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_tacker master: Modify the etc directory in default/main  https://review.openstack.org/53885509:32
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Fix memcache caching w/ multiple cache servers  https://review.openstack.org/53331409:40
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [WIP] Add ansible modules idempotency testing  https://review.openstack.org/53885609:42
*** shardy has joined #openstack-ansible09:43
*** vnogin has joined #openstack-ansible09:46
*** vnogin has quit IRC09:46
*** cjloader has joined #openstack-ansible09:48
*** cjloader has quit IRC09:52
odyssey4meo/10:02
*** bauruine has quit IRC10:07
*** bauruine has joined #openstack-ansible10:08
evrardjphello odyssey4me10:13
bndzor_hmm im not having the best performance with ceph and i cant figure out why.. is there anything i should tune ?10:28
evrardjpbndzor_: we are simply using ceph ansible, maybe there are setting in ceph ansible you can use10:32
evrardjpelse I'd say, check your network stack and all.10:33
*** threestrands_ has joined #openstack-ansible10:33
*** threestrands has quit IRC10:36
*** vnogin has joined #openstack-ansible10:37
*** jappleii__ has joined #openstack-ansible10:37
*** jappleii__ has quit IRC10:38
*** jappleii__ has joined #openstack-ansible10:39
*** Neptu_ has joined #openstack-ansible10:39
*** threestrands_ has quit IRC10:41
*** vnogin has quit IRC10:42
evrardjpodyssey4me: hey, maybe rcarrillocruz got a present for you10:42
*** rcarrillocruz has joined #openstack-ansible10:44
* rcarrillocruz waves10:44
evrardjphello rcarrillocruz !10:45
evrardjpI just pinged odyssey4me , telling him you have presents10:46
evrardjpbecause he is a nodepool user he will like it :p10:46
evrardjpAt least, I think.10:46
evrardjpoh whie you are here rcarrillocruz could you check this for me please? https://github.com/ansible/ansible/pull/3505310:47
evrardjpbetween many pr, it just needs a shipit :)10:48
*** cjloader has joined #openstack-ansible10:48
openstackgerritJames E. Blair proposed openstack/openstack-ansible-galera_server master: Zuul: Remove project name  https://review.openstack.org/53886810:49
odyssey4meevrardjp yep - discussing it in #zuul10:50
evrardjpoh I see... too many channels, I am lost!10:51
evrardjplet me join #zuul10:51
*** cjloader has quit IRC10:53
-openstackstatus- NOTICE: Zuul is currently under heavy load. Do not *recheck* or *approve* any changes.11:02
*** electrofelix has joined #openstack-ansible11:06
openstackgerritJames E. Blair proposed openstack/openstack-ansible-galera_server stable/newton: Zuul: Remove project name  https://review.openstack.org/53887611:09
*** vnogin has joined #openstack-ansible11:10
openstackgerritJames E. Blair proposed openstack/openstack-ansible-galera_server stable/ocata: Zuul: Remove project name  https://review.openstack.org/53888311:29
*** stuartgr has joined #openstack-ansible11:33
*** cjloader has joined #openstack-ansible11:48
openstackgerritJames E. Blair proposed openstack/openstack-ansible-galera_server stable/pike: Zuul: Remove project name  https://review.openstack.org/53888911:49
*** cjloader has quit IRC11:52
*** zenirc369 has joined #openstack-ansible11:58
zenirc369How to use grouped/combined register variable output values [values can be repeated/duplicate] of multiple hosts?11:59
zenirc369https://stackoverflow.com/questions/48499649/how-to-use-grouped-combined-register-variable-output-values-values-can-be-repea11:59
*** vnogin has quit IRC12:02
*** vnogin has joined #openstack-ansible12:03
evrardjpzenirc369: combine filter combines dicts12:17
evrardjpzenirc369: + operator concats lists12:17
evrardjpmath operators should do the trick too12:17
evrardjpmath filters*12:17
evrardjplike | unique12:17
evrardjpof difference12:17
evrardjpI don't know what you are looking for but grouping seems that12:18
evrardjpalso if you're expecting always the same answer etc. maybe run_once can help.12:18
evrardjpnot sure what the issue is, so giving you a generic feedback zenirc36912:19
evrardjphttp://docs.ansible.com/ansible/latest/playbooks_filters.html#set-theory-filters for the set theory filters12:19
zenirc369filtering is fine12:20
zenirc369I have idea about filtering the list12:21
zenirc369but at first I need all the cidr values to be grouped to form a list12:21
*** sxc731 has quit IRC12:23
evrardjpI am not sure what the problem is.12:24
evrardjpIf your variable is a host var, it's meant for being scoped as a host var12:24
evrardjpif it's a group_var, then any host in the group will have the same content12:25
evrardjp(unless usual ansible overrides)12:25
evrardjpit all depends on your case12:25
evrardjpthink about set fact, delegation, etc.12:25
evrardjpand think about your structure of data12:25
evrardjp(and where your overrides are)12:26
evrardjpbut this isn't openstack-ansible specific, is it?12:26
*** sxc731 has joined #openstack-ansible12:27
*** vnogin has quit IRC12:34
*** vnogin has joined #openstack-ansible12:41
*** vnogin has quit IRC12:42
zenirc369the environment I'm trying to implment is on openstack12:44
*** cjloader has joined #openstack-ansible12:48
zenirc369evrardjp: thnks for your inputs :)12:49
*** armaan_ has joined #openstack-ansible12:49
*** vnogin has joined #openstack-ansible12:51
*** armaan has quit IRC12:52
*** cjloader has quit IRC12:52
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: Vagrantfile: Use rsync to copy working directory to VM  https://review.openstack.org/53890312:56
armaan_evrardjp: hello, Does this https://review.openstack.org/#/c/538407/ mean that i can start testing O->P upgrade? :)13:02
evrardjparmaan_: nope that was for P-Q role test upgrade13:02
evrardjparmaan_: https://review.openstack.org/#/c/538315/ is the one for O-P13:03
evrardjpI have yet to test it manually too though, so I'd wait.13:03
evrardjpIt should work.13:03
armaan_evrardjp: gotcha! thanks for letting me know :)13:03
evrardjpand I think i tested it last week, but it was long ago.13:03
evrardjp:D13:03
armaan_ohh okk :)13:04
evrardjparmaan_: oh I tested it this morning to make sure. I got 91% api downtime for glance13:05
evrardjpduring the process13:05
armaan_eeehhh ... 91%13:05
*** zenirc369 has quit IRC13:06
evrardjparmaan_: http://paste.openstack.org/show/656884/13:06
evrardjpbut that's a major upgrade too.13:06
evrardjpso it succeeded13:06
evrardjpand that's api downtime, maybe not that important to you.13:06
evrardjpBut I will have a look in the future to reduce that downtime value13:07
evrardjpanyway, with that patch in, it should be better.13:07
armaan_evrardjp: we will inform our customers about the service window and after that i think we can live with any downtime :)13:07
evrardjpwhat a luxury :)13:08
evrardjpI think the most important/worrying part on your case is magnum...13:08
armaan_evrardjp: hehe ... yes.13:08
armaan_evrardjp: How can i remove the magnum service properly before upgrading?13:09
armaan_evrardjp: Destroy containers, remove from inventory?13:09
*** hamza21 has quit IRC13:15
*** vnogin has quit IRC13:16
evrardjpyes13:19
evrardjpprobably remove the env.d or conf.d overrides if any13:19
odyssey4meremove the endpoint too, I expect?13:21
odyssey4meendpoint(s)13:21
armaan_evrardjp odyssey4me: Roger that. thanks!13:22
*** woodard has quit IRC13:26
*** woodard has joined #openstack-ansible13:27
evrardjpodyssey4me: armaan_ yeah probably worth cleaning up tenants, dbs, and stuff like that13:28
evrardjpcloudnull: are you there?13:28
armaan_evrardjp: oh, you mean removing service user from service tenant etc13:29
evrardjpyes if need be. But that's maybe extra cleanup13:30
evrardjpas usual, take a backup first :p13:30
-openstackstatus- NOTICE: Zuul is currently under heavy load. Do not *recheck* or *approve* any changes until we give the go ahead.13:30
*** ChanServ changes topic to "Zuul is currently under heavy load. Do not *recheck* or *approve* any changes until we give the go ahead."13:30
evrardjpbut... my patches!13:31
evrardjp:D13:31
*** vnogin has joined #openstack-ansible13:34
mnaserso, RBD specific, OSA doesn't have anything that does active/passive cinder-volume processes, multiple processes with the same "host" is not recommended by the cinder team13:46
mnaseri know tripleo uses pacemaker to do active/passive.  it might be a bit overkill to run pacemaker just for this (or is it not?)13:47
mnaserlogan-: ^13:47
odyssey4menope, we don't do anything like that13:47
odyssey4meeach cinder-volume host is its own13:47
mnaserodyssey4me: the thing i have with that is that if a specific cinder-volume service goes down for some reason, technically the service is still operational but all volumes become un-managable13:47
*** cjloader has joined #openstack-ansible13:48
odyssey4meyes, that is true13:48
odyssey4meIIRC cinder-manage now has a command to migrate the management over to another cinder-volume instance13:48
mnaserso any maintenance on controllers can cause a control plane outage (unless you start messing around the db)13:48
mnaseroh TIL, let me read about that13:48
*** vnogin has quit IRC13:49
mnasercinder-manage volume update_host --currenthost <current host> --newhost <new host>13:49
mnaserUpdates the host name of all volumes currently associated with a specified host.13:49
mnaserindeed13:49
*** vnogin has joined #openstack-ansible13:49
openstackgerritJames E. Blair proposed openstack/openstack-ansible-repo_build master: Zuul: Remove project name  https://review.openstack.org/53892613:50
odyssey4meyep, so instead of adding more moving parts we prefer to think that if an admin can't get the service back up the admin can use that command instead13:50
mnaserodyssey4me: fair thinking13:50
mnaserthat's a pretty clean mitigation, didnt know about that command13:51
*** cjloader has quit IRC13:53
evrardjpmaybe that should be part of ops docs.13:57
mgariepymorning everyone14:00
evrardjpmorning mgariepy14:00
mgariepyhow are things going this morning ?14:01
evrardjpzuul is in a bad shape, see chan title14:02
evrardjpI hope my patch to freeze stuff for m3 will pass, I am already late14:03
evrardjpthen unfreeze and here we go again!14:03
*** RandomTech has joined #openstack-ansible14:03
openstackgerritJames E. Blair proposed openstack/openstack-ansible-repo_build stable/newton: Zuul: Remove project name  https://review.openstack.org/53892814:10
evrardjpxdfil: I asked on openstack-charms, no frozen uid/gid for their ubuntu packages, so we can probably implement Red Hat/IBM's uids/gids.14:14
*** dmsimard is now known as dmsimard|afk14:17
*** pabelanger has joined #openstack-ansible14:24
pabelangermorning!14:24
pabelangercan you please top approving patches14:24
*** cmart has joined #openstack-ansible14:24
pabelangerspecifically status:open+topic:zuulv3-projects14:24
pabelangerwe are running into memory issues and likely need to schdule those reviews for approval14:25
evrardjptop approving or stop approving?14:26
evrardjppabelanger: I haven't rechecked/+W patches since the message, not sure about others.14:27
evrardjppabelanger: how can we help?14:27
-openstackstatus- NOTICE: we've been able to restart zuul, and re-enqueue changes for gate. Please hold off on recheck or approves, we are still recovering. More info shortly.14:28
pabelangerevrardjp: if you don't mind reaching out to team mates using the following topic: https://review.openstack.org/#/q/status:open+topic:zuulv3-projects14:29
pabelangercan see some openstack ansible patches approved14:29
pabelangersadly, this isn't the best way to handle it, but until we clear patches already approved we run the risk of running out of memory again14:30
openstackgerritJames E. Blair proposed openstack/openstack-ansible-repo_build stable/ocata: Zuul: Remove project name  https://review.openstack.org/53893114:30
odyssey4meit might be by someone who's not yet looking at IRC - maybe just browsing on their phone14:30
odyssey4meit also seems like jeblair has new patch submissions being done at some sort of interval14:30
mnaserodyssey4me: every 20 minutes14:30
pabelangeryes14:31
pabelangerworking on stopping that too14:31
pabelangermnaser: if you also don't mind approving puppet changes, that would be helpful14:32
evrardjpnot sure I understood, you mean they shouldn't go in, and we should -2 right now?14:32
evrardjpor it's the other way around?14:32
pabelangerno, things approved is fine for now. Already loaded into memory14:32
evrardjpbecause the patches approved on this topic were approved at 9:4014:32
evrardjpok14:32
pabelangerbut +W is what we need to hold off on for the moment14:33
evrardjpwell, we'll just say to stay calm until the storm is over.14:33
pabelangeryah14:33
evrardjpyeah ok understood.14:33
pabelangerthanks14:33
*** gkadam has quit IRC14:33
evrardjphwoarang: mhayden mgariepy andymccr odyssey4me cloudnull see the message just here: do not recheck or +w please, until further infra notice! Thanks!14:34
*** dave-mccowan has joined #openstack-ansible14:36
*** zenirc369 has joined #openstack-ansible14:39
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: tasks: Ensure Open vSwitch is started for all providers that need it  https://review.openstack.org/53893314:40
*** santacloud has joined #openstack-ansible14:41
*** zenirc369 has quit IRC14:44
*** SerenaFeng has joined #openstack-ansible14:46
*** zenirc369 has joined #openstack-ansible14:47
openstackgerritMerged openstack/openstack-ansible-os_horizon stable/ocata: Zuul: Remove project name  https://review.openstack.org/53875714:47
dcdamienhttps://review.openstack.org/#/c/538056/414:48
dcdamienmhayden: can you elaborate?14:48
*** indistylo has joined #openstack-ansible14:48
openstackgerritJames E. Blair proposed openstack/openstack-ansible-repo_build stable/pike: Zuul: Remove project name  https://review.openstack.org/53893514:50
*** vnogin has quit IRC14:50
*** esberglu has joined #openstack-ansible14:51
*** vnogin has joined #openstack-ansible14:51
*** cjloader has joined #openstack-ansible14:53
RandomTechHey, can someone explain to me how external networking is determined in terms of the playbooks?14:53
evrardjpwhat do you mean?14:54
RandomTechone moment while i think of how to word this please, sorry14:56
*** cjloader has quit IRC14:57
*** cjloader has joined #openstack-ansible14:57
RandomTechin openstack_user_config, which network is normally used for the creation of floating ips. or how do i make an extra network for that purpose14:58
*** hw_wutianwei has joined #openstack-ansible14:58
evrardjpdcdamien: answered for him14:58
evrardjpRandomTech: understood the question14:59
evrardjpso14:59
evrardjpthe provider_networks can arrive to your neutron agent containers.14:59
evrardjpRandomTech: for example: https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.aio#L9315:00
evrardjpor https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.aio#L8415:00
evrardjpin this case: https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.aio#L76-L8415:00
evrardjpyou are passing a host bind override (a host nic directly) named eth12, to the container that can then use that eth12 nic.15:01
evrardjpbecause the type is flat, we are generating all you need to use it as a flat network (which means no .1q tagging)15:01
evrardjpthis: https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.aio#L85-L93 is different15:01
openstackgerritMerged openstack/openstack-ansible-os_tacker stable/pike: Zuul: Remove project name  https://review.openstack.org/53840415:02
openstackgerritMerged openstack/openstack-ansible-os_tacker master: Zuul: Remove project name  https://review.openstack.org/53840115:02
evrardjpit allows the container to have a nic that's allowing .1q traffic (i.e vlans)15:02
evrardjpwhose VLAN ID is matching the "range" stanza.15:03
*** sxc731 has quit IRC15:03
evrardjpto be more specific, there is a nic that's passed to the container (eth11), and on that nic you can use .1q for neutron.15:03
RandomTechso groupbind: neutron_linuxbridge_agent allows for the container to attach to that network. host_bind_override passes a nic from the host directly to the container. net_type tells it if its flat or not?15:05
*** SerenaFeng has quit IRC15:06
RandomTechwould i use ip_from_q if i was wanting to assign floating evrardjp ?15:07
evrardjpnope.15:09
evrardjpafter the deploy of openstack, you should use the openstack cli to create your networks15:09
RandomTechokay. would you mind if i show you what i typed after i try configuring one?15:10
RandomTechim pretty new to openstack in general15:10
jrosserRandomTech: here is a little thing we made to turn a fresh openstack "useful" https://github.com/openstack/openstack-ansible-ops/blob/master/multi-node-aio/playbooks/openstack-service-setup.yml15:10
evrardjpI am not sure if I can help, but I am pretty sure someone might help you here.15:10
cloudnullmornings15:11
evrardjpcloudnull: good morning!15:11
evrardjpplease don't do patches, recheck patches or approve patches until infra is feeling better :)15:11
cloudnullha ok15:11
evrardjpcloudnull: do you mind, in the meantime, help me on a small challenge?15:11
* cloudnull goes to recheck everything :)15:12
evrardjpbypassing stupid pylint is the challenge15:12
cloudnullsure what's up ?15:12
RandomTechevrardjp: this is what i came up with: http://paste.openstack.org/show/657041/15:12
*** gus has quit IRC15:13
RandomTechassuming i make a br-external of course15:13
evrardjpcloudnull: let's talk on pm, easier to not pollute this chan15:13
*** gus has joined #openstack-ansible15:14
*** vnogin has quit IRC15:20
openstackgerritMerged openstack/openstack-ansible-os_magnum stable/ocata: Zuul: Remove project name  https://review.openstack.org/53863415:23
*** indistylo has quit IRC15:27
*** bauruine has quit IRC15:28
*** bauruine has joined #openstack-ansible15:34
*** sxc731 has joined #openstack-ansible15:44
openstackgerritMerged openstack/openstack-ansible-os_sahara stable/pike: Zuul: Remove project name  https://review.openstack.org/53838615:45
openstackgerritMerged openstack/openstack-ansible-os_sahara stable/ocata: Zuul: Remove project name  https://review.openstack.org/53838515:45
openstackgerritMerged openstack/openstack-ansible-os_sahara stable/newton: Zuul: Remove project name  https://review.openstack.org/53838415:45
openstackgerritMerged openstack/openstack-ansible-os_horizon stable/newton: Zuul: Remove project name  https://review.openstack.org/53875415:45
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Zuul: Remove project name  https://review.openstack.org/53874915:45
evrardjpit's raining merges!15:45
*** esberglu has quit IRC15:48
*** esberglu has joined #openstack-ansible15:49
*** phalmos has joined #openstack-ansible15:57
RandomTechevrardjp: did you get a chance to look at the paste i did? sorry i had to try to fix my boss's mac15:59
*** vnogin has joined #openstack-ansible16:00
*** hw_wutianwei has quit IRC16:01
*** vnogin has quit IRC16:04
*** vnogin has joined #openstack-ansible16:04
*** santacloud has quit IRC16:05
*** jappleii__ has quit IRC16:05
RandomTechor could you take a look cloudnull?16:05
*** phalmos has quit IRC16:11
*** woodard has quit IRC16:18
*** ChanServ changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs"16:19
-openstackstatus- NOTICE: zuul.o.o is back online, feel free to recheck / approve patches.16:19
cloudnullRandomTech: I'd set the config to something like so: http://paste.openstack.org/show/65714316:19
cloudnullRandomTech: and have the interface file to be something like http://paste.openstack.org/show/65714116:19
*** vnogin has quit IRC16:19
cloudnullthat would create a "br-external" bridge and hang an eth5 interface off of the bridge allowing neutron to use a flat network16:21
jrossercloudnull: does that need an IP on the bridge?16:21
cloudnullit does not need an IP.16:21
jrosserok, what i really meant was there was one in the paste and i was puzzled16:21
cloudnullthe IP would only be useful if you wanted to have access to the instances from the physical host.16:21
jrosserright, i see16:22
cloudnullif you have multiple subnets that you were going to use for flat networks then it wouldn't be all that usefule .16:22
cloudnull**useful16:22
*** chyka has joined #openstack-ansible16:22
*** bauruine has quit IRC16:22
*** chyka has quit IRC16:22
*** bauruine has joined #openstack-ansible16:23
*** chyka has joined #openstack-ansible16:23
RandomTechcloudnull: why net_name: flat? and were just wanting this as a network to assign floating ips on16:24
*** pcaruana has quit IRC16:25
RandomTechand what interface file are you talking about? the hosts?16:26
cloudnullthe netname is used within neutron16:27
RandomTechso that would allow me to see it in the interface when creating networks?16:28
*** phalmos has joined #openstack-ansible16:30
RandomTechcloudnull: why is it bridging to br-vlan-veth?16:31
cloudnullopps line 17 was a typo16:35
cloudnullthat should have been br-external-veth16:35
*** armaan has joined #openstack-ansible16:36
*** sxc731 has quit IRC16:38
*** armaan_ has quit IRC16:39
RandomTechthis seems alot more complicated than the other bridges16:39
cloudnullit is a bit more complex but the setup has been working for us for quite a while now.16:40
cloudnullit's effectively just create an available interface so neutron can bind ot16:41
cloudnull**tp16:41
cloudnullhahahah.16:41
cloudnullI can't type16:41
cloudnullmind sharing the br-external interface config you have now ?16:41
RandomTechi was just setting this up one moment while i type something up16:42
cloudnullok.16:42
*** phalmos has quit IRC16:42
*** sxc731 has joined #openstack-ansible16:47
RandomTechthis is my template file for the interface files cloudnull: http://paste.openstack.org/show/657192/16:48
RandomTechso far at least16:49
cloudnullI'd add "offload-sg off" to all of the bridges16:50
cloudnulland change the br-external to look like http://paste.openstack.org/show/657197/16:50
cloudnullotherwise the rest looks good16:51
*** Leo_m has joined #openstack-ansible16:51
RandomTechwhy line 5-7?16:51
cloudnullthat's what creates the "eth5" interface16:52
cloudnullwhich neutron will bind to for the flat network type16:52
cloudnulltechnically that would only need to be done on compute nodes16:52
RandomTechcould i just have it bind to the existing public interface?16:52
cloudnull** compute and network nodes16:53
RandomTechthe enp7s0f116:53
cloudnullyes you could have it just bind to that, if you set the hostbind override to that address and that address is the same across the fleet.16:53
cloudnulls/address/interface/16:54
RandomTechit should be16:54
*** ndusek has joined #openstack-ansible16:55
ndusekhey all16:55
cloudnullwhen using a flat network, neutron will attempt to create bridges on a given interface, so if nothing else is bridging that interface it'll work great.16:56
cloudnullhanging the veth off a br-external interface is just a way to ensure an open interface is available but there are a lot of ways to get that done.16:56
RandomTechso like this? http://paste.openstack.org/show/657205/16:56
cloudnullo/ ndusek16:56
ndusekI just deployed openstack ansible all-in-one last week. Had to do a reboot today and none of my containers came back and I can't restart them16:57
cloudnullRandomTech: yes16:57
RandomTechthe only thing bridging off of that interface is the br-external i just created in the template16:57
ndusekany clues as to what might be going on here?16:57
ndusekI can provide lxc log info16:57
cloudnullon the compute nodes, the flat network type will run on "enp7s0f1" and within the containers neutron will run on "eth5" which will all be part of the same l2.16:58
cloudnullRandomTech: will that template be run on compute nodes too? if so then you might need to make the br-external interface conditional.16:59
RandomTechthe template is run on all of the host16:59
RandomTechcurrently anyway16:59
RandomTechi used it to help do preliminary setup before running openstack playbooks17:00
*** kstev has joined #openstack-ansible17:01
cloudnullRandomTech: http://paste.openstack.org/show/65721017:01
cloudnullso if the node is not in the compute group it'll create the bridge17:02
cloudnullotherwise it'll just rely on the interface which is defined earlier in your template.17:02
RandomTechthat works with the template command in ansible?17:03
*** rstarmer has joined #openstack-ansible17:03
cloudnullyes17:03
cloudnullthe template module is effectively the copy module with jinja2 templating17:04
RandomTechim not sure how to integrate this with the openstack-ansible inventory honestly17:04
RandomTechwe were just running a ansible script sepearately on the bare metal17:06
cloudnulli'd probably continue running that with an external inventory.17:08
cloudnullyou could however add the needed options to the openstack_user_config.yml17:08
RandomTechthe external inventory doesnt have a group['nova_compute']17:09
cloudnulllooks like you added a hash for "openstack_storage_ip"17:09
RandomTechheres an example of one of the computes /etc/network/interfaces file after we ran it (pre adding external stuff): http://paste.openstack.org/show/657221/17:12
*** akasurde has joined #openstack-ansible17:14
cloudnullRandomTech: http://paste.openstack.org/show/657228/17:15
cloudnullif you wanted to extend the osa inventory with the options needed for your template ^17:15
*** phalmos has joined #openstack-ansible17:15
cloudnullRandomTech: to make that work in your template all you'd need to change is "{{ openstack_storage_ip[bare_metal[inventory_hostname]] }}" to "{{ openstack_storage_ip['bare_metal'] }}"17:17
*** woodard has joined #openstack-ansible17:17
*** akasurde has quit IRC17:17
RandomTechwould is cause a problem if the computes have a br-external?17:19
RandomTechsorry for all the questions, my boss basically told me set up openstack with no prior knowledge of it17:25
RandomTechall i knew was that it was cluster computing17:25
*** guhcampos has joined #openstack-ansible17:25
*** guhcampos_ has joined #openstack-ansible17:25
*** phalmos has quit IRC17:26
*** armaan_ has joined #openstack-ansible17:29
xdfilevrardjp: cool, i've been thinking about the optimal way to handle it. currently if you set a uid value OSA will try to change it but if the existing uid has any running processes it will fail and leave you with a small mess to clean up17:29
xdfilwell it could be a big mess if it is in production17:30
*** sxc731 has quit IRC17:30
*** armaan_ has quit IRC17:31
*** mbuil has quit IRC17:31
*** armaan_ has joined #openstack-ansible17:32
*** armaan has quit IRC17:32
*** cjloader has quit IRC17:33
*** cjloader has joined #openstack-ansible17:33
*** cjloader has quit IRC17:39
*** cjloader has joined #openstack-ansible17:40
cloudnullRandomTech: if the compute nodes have "br-external" then you'll need to hang the veth off that bridge17:40
*** cjloader has quit IRC17:41
*** cjloader has joined #openstack-ansible17:42
*** cjloader has quit IRC17:45
*** cjloader has joined #openstack-ansible17:46
RandomTechdo the infrastucture or storage nodes need the bridge cloudnull ?17:46
cloudnullyes.17:48
*** cjloader has quit IRC17:48
*** cjloader has joined #openstack-ansible17:48
cloudnullthe exception would be network and compute17:48
RandomTechhow will the routing work if we hange the veth?17:48
cloudnulleverything will pass through the bridege17:48
RandomTechit looks like well have to do that as our storage and compute are combined17:49
cloudnullso eth5 > br-external > physical17:49
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_glance master: Set glance images dir owners each time  https://review.openstack.org/53318917:49
RandomTechso just use this for the bridge in my template? : http://paste.openstack.org/show/657197/17:49
*** peri has quit IRC18:02
*** phalmos has joined #openstack-ansible18:03
*** RandomTech has quit IRC18:05
*** phalmos has quit IRC18:09
*** stuartgr has quit IRC18:29
xdfilwhere is OSA saving iptables stuff at?18:40
xdfilthere are firewall rules, but I don't see where they are getting loaded from18:46
*** kstev has quit IRC18:46
cloudnullxdfil: if it's an openstack firewall rule it'd come from nova / neutron18:56
cloudnullwe don't do much in osa for firewall rules, outside of the aio18:56
*** shardy has quit IRC18:56
*** ivve has quit IRC18:57
xdfilthanks19:00
*** poopcat has joined #openstack-ansible19:02
*** ndusek has left #openstack-ansible19:04
openstackgerritMerged openstack/openstack-ansible master: Update all SHAs for 17.0.0.0b3  https://review.openstack.org/53868019:05
*** poopcat1 has joined #openstack-ansible19:06
*** poopcat has quit IRC19:08
*** cjloader has quit IRC19:08
*** cjloader has joined #openstack-ansible19:09
*** ivve has joined #openstack-ansible19:11
*** phalmos has joined #openstack-ansible19:12
*** hamza21 has joined #openstack-ansible19:14
*** mgariepy has quit IRC19:15
*** rstarmer has quit IRC19:17
*** rstarmer has joined #openstack-ansible19:19
*** mgariepy has joined #openstack-ansible19:24
*** john51 has quit IRC19:30
*** john51 has joined #openstack-ansible19:31
*** phalmos has quit IRC19:37
*** rstarmer has quit IRC19:39
*** phalmos has joined #openstack-ansible19:39
*** rstarmer has joined #openstack-ansible19:39
*** hamza21 has quit IRC19:47
*** phalmos has quit IRC19:53
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_nova master: Fix SELinux file contexts for nova's ssh keys  https://review.openstack.org/53489119:55
*** phalmos has joined #openstack-ansible19:57
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_trove master: Updated from global requirements  https://review.openstack.org/53507419:59
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_nova master: Adjust deprecated nova console variables  https://review.openstack.org/53283620:01
*** ivve has quit IRC20:03
*** pramodrj07 has joined #openstack-ansible20:04
*** hybridpollo has joined #openstack-ansible20:05
*** aludwar has joined #openstack-ansible20:05
*** aludwar has quit IRC20:06
*** aludwar has joined #openstack-ansible20:07
*** aludwar has quit IRC20:11
*** hybridpollo has quit IRC20:11
*** hybridpollo has joined #openstack-ansible20:12
*** ivve has joined #openstack-ansible20:15
*** rstarmer has quit IRC20:17
mgariepyany cores to push this one over ? https://review.openstack.org/#/c/538324/20:17
openstackgerritMerged openstack/openstack-ansible-rsyslog_server master: Zuul: Remove project name  https://review.openstack.org/53862020:31
*** kstev has joined #openstack-ansible20:54
*** pbandark has joined #openstack-ansible21:05
openstackgerritMerged openstack/openstack-ansible-os_glance master: Set glance images dir owners each time  https://review.openstack.org/53318921:15
cloudnullmgariepy: done.21:17
mgariepythanks cloudnull21:18
cloudnullanytime !21:19
*** kstev1 has joined #openstack-ansible21:27
openstackgerritMarc GariĆ©py (mgariepy) proposed openstack/openstack-ansible-lxc_hosts master: Change default for sshd to only all keybased auth  https://review.openstack.org/53902021:27
mgariepymhayden, cloudnull, ^^21:27
*** kstev has quit IRC21:28
cloudnulli'm good with that21:29
mgariepyon this cya tomorrow.21:30
cloudnulltake care mgariepy21:30
*** poopcat1 is now known as poopcat21:37
mhaydenmgariepy: ol' evrardjp makes a valid point on a reno21:41
mhaydeneven though people shouldn't be using passwords by default, it's worth mentioning21:42
*** woodard has quit IRC21:43
idlemindrunning stable/pike on a freshly installed centos 7 host and i failed on lxc_hosts get init.scope pids.max value. the log says "file not found: /sys/fs/cgroup/pids/init.scope/pids.max"21:56
idlemindi verified it doesn't exist, any thoughts on how i should correct it or is it a false positive?21:56
cloudnullidlemind: looks like we use a block / rescue there22:01
idlemind^^ ahh fix is in upstream openstack-ansible-lxc_hosts (master) but not in their stable/pike ... seems older versions of systemd don't support init.scope so the patch to fix it basically skips the task that failed. i'll ignore for now and watch for the patch to move into stable/pike22:01
idlemindhttps://github.com/openstack/openstack-ansible-lxc_hosts/commit/165bd142b1d40467284f6b13457888b599b2bebd22:01
cloudnullidlemind: https://github.com/openstack/openstack-ansible-lxc_hosts/blob/stable/pike/tasks/lxc_post_install.yml#L74-L11722:02
cloudnullit should have been caught by the rescue and printed out the debug info22:02
cloudnullhowever there's a really poor user experience there22:03
cloudnullit'll show that the task failed22:03
cloudnullhowever the playbook succeeds22:03
cloudnullwe recently added https://review.openstack.org/#/c/530959/22:03
cloudnullwhich will notify the user if the playbook failed by looking at the return code22:04
cloudnullbecause of this issue.22:04
cloudnullthe UX however should imporve in ansible 2.5, but we're not there yet.22:04
idlemindcloudnull yup i see the additional debug in green below the failed as well good to see additional pretty print information as well22:04
idlemindcloudnull also seems the "fix" is upstream in the sub repo so it just has to get cherry picked into stable/pike for centos 722:05
cloudnull** This is all assuming the task failed, was caught by the rescue and everything else succeeded.22:05
idlemindor the right sha adjusted in openstack-ansible for osa-lxc_hosts however it is linked22:05
*** threestrands has joined #openstack-ansible22:05
*** threestrands has quit IRC22:05
*** threestrands has joined #openstack-ansible22:05
cloudnulllets go see maybe, maybe it's in the branch but not in a stable tag yet22:05
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_hosts stable/pike: tasks: lxc_post_install: Handle missing init.scope scope  https://review.openstack.org/53902622:07
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible stable/pike: Add a return code check to the osa wrapper  https://review.openstack.org/53902722:07
cloudnullyup, both needed to be cherry-picked22:07
idlemindspeedy speedy22:07
*** threestrands_ has joined #openstack-ansible22:07
cloudnullso you can cherry-pick those into your checkout for now22:08
cloudnulland as soon as they're merged they'll be included in the next stable tag22:08
*** threestrands has quit IRC22:08
*** threestrands_ has quit IRC22:08
cloudnullif you have time to review either of those pr's it'd be greatly appreciated.22:08
*** threestrands has joined #openstack-ansible22:08
*** threestrands has quit IRC22:08
*** threestrands has joined #openstack-ansible22:08
idlemindcloudnull any tips on how i'd do that? just cherry-pick them on my side and post feedback?22:10
cloudnullcd to the dir w/ the role "/etc/ansible/roles/lxc_hosts" and run `git fetch https://git.openstack.org/openstack/openstack-ansible-lxc_hosts refs/changes/26/539026/1 && git checkout FETCH_HEAD`22:10
idlemindsweet thx22:11
cloudnullin gerrit in the top right corner there's the "download" button22:11
cloudnullyou can see a couple options for cherry-pick22:11
idlemindrunning another playbook atm so when that finishes i'll grab the change and re-run to see if it works22:11
cloudnullcheckout, etc22:11
cloudnullcool!22:11
idlemindnice didn't know the download thing had the cheater commands22:12
*** rstarmer has joined #openstack-ansible22:16
*** threestrands_ has joined #openstack-ansible22:21
*** threestrands has quit IRC22:23
*** ndusek has joined #openstack-ansible22:26
openstackgerritMerged openstack/openstack-ansible-os_cloudkitty master: Zuul: Remove project name  https://review.openstack.org/53861022:27
*** woodard has joined #openstack-ansible22:29
*** dave-mccowan has quit IRC22:31
*** Leo_m_ has joined #openstack-ansible22:32
*** Leo_m has quit IRC22:35
*** jappleii__ has joined #openstack-ansible22:35
*** jappleii__ has quit IRC22:36
*** jappleii__ has joined #openstack-ansible22:37
*** Leo_m_ has quit IRC22:37
*** threestrands_ has quit IRC22:37
cloudnullidlemind: yup.22:40
* cloudnull loves me some cheater commands22:40
cloudnullI'm headed off , take care all22:41
idlemindnn22:41
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Zuul: Remove project name  https://review.openstack.org/53852422:41
*** rstarmer has quit IRC22:47
*** ndusek has quit IRC22:49
*** phalmos_ has joined #openstack-ansible22:56
*** kstev has joined #openstack-ansible22:57
*** kstev1 has quit IRC22:57
*** rstarmer has joined #openstack-ansible22:58
*** phalmos has quit IRC22:59
openstackgerritMerged openstack/openstack-ansible-plugins master: Zuul: Remove project name  https://review.openstack.org/53861323:01
*** pmannidi has joined #openstack-ansible23:01
*** kstev has quit IRC23:08
*** rstarmer has quit IRC23:26
*** zenirc369 has quit IRC23:29
*** zenirc369 has joined #openstack-ansible23:33
*** cjloader has quit IRC23:38
*** pbandark has quit IRC23:42
*** logan- has quit IRC23:45
*** errr has quit IRC23:45
*** Oku_OS-away has quit IRC23:45
*** xar- has quit IRC23:45
*** jillr has quit IRC23:45
*** logan- has joined #openstack-ansible23:51
*** errr has joined #openstack-ansible23:51
*** Oku_OS-away has joined #openstack-ansible23:51
*** xar- has joined #openstack-ansible23:51
*** jillr has joined #openstack-ansible23:51
openstackgerritMerged openstack/openstack-ansible stable/pike: Remove default_bind_mount_logs from config changes.  https://review.openstack.org/53832423:52
*** dave-mccowan has joined #openstack-ansible23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!