Monday, 2020-04-06

*** velmeran has quit IRC02:03
*** evrardjp has quit IRC04:36
*** evrardjp has joined #openstack-ansible04:37
*** udesale has joined #openstack-ansible04:51
*** udesale has quit IRC04:52
*** udesale has joined #openstack-ansible04:52
*** udesale has quit IRC05:03
*** udesale has joined #openstack-ansible05:03
*** shyamb has joined #openstack-ansible05:12
*** shyamb has quit IRC05:43
*** shyamb has joined #openstack-ansible05:55
*** miloa has joined #openstack-ansible06:09
*** sep has joined #openstack-ansible06:25
*** jbadiapa has joined #openstack-ansible06:27
*** bjolo_ has joined #openstack-ansible06:29
*** rpittau|afk is now known as rpittau06:35
*** kleini has joined #openstack-ansible06:49
*** shyam89 has joined #openstack-ansible06:55
*** yolanda has joined #openstack-ansible06:56
*** shyamb has quit IRC06:58
*** shyam89 has quit IRC07:11
*** shyamb has joined #openstack-ansible07:43
*** this10nly has joined #openstack-ansible07:48
*** tosky has joined #openstack-ansible07:58
*** spsurya_ has joined #openstack-ansible07:59
*** sshnaidm|afk is now known as sshnaidm08:07
*** shyamb has quit IRC08:16
*** shyamb has joined #openstack-ansible08:17
*** DanyC has joined #openstack-ansible08:17
*** shyamb has quit IRC09:21
*** shyamb has joined #openstack-ansible09:22
*** andrewbonney has joined #openstack-ansible09:44
*** shyamb has quit IRC10:06
*** shyamb has joined #openstack-ansible10:17
*** ierdem has joined #openstack-ansible10:56
*** ut2k3 has joined #openstack-ansible10:57
ierdemHello guys, i have a question about resizing a running VM. I tried to increase vCPU counts of a running VM and after that i connected succesfully but it is too slow to run anything. When i see process list via "top" command, i saw CPU usage is approximately 1000 percent and it  decrease sometimes but increase again. Any suggestions please?10:58
ut2k3Hi guys :) quick question about the process. We started back then unfortunately with one controller node and want now to move on 3 of them. Whats the best Upgrade process here? Is the ansible taking care of all steps? So "just" adding the new ones to _infrastructure_hosts or more steps to think about?11:00
*** shyamb has quit IRC11:02
*** aedc has quit IRC11:11
*** shyamb has joined #openstack-ansible11:17
*** jamesdenton_wfh has joined #openstack-ansible11:26
*** rh-jelabarre has joined #openstack-ansible11:27
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-repo_build master: Retire openstack-ansible-repo_build role  https://review.opendev.org/71638911:30
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-pip_install master: Retire openstack-ansible-repo_build role  https://review.opendev.org/71771711:35
*** DanyC has quit IRC11:44
*** shyamb has quit IRC11:44
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_tempest master: Update used cirros image  https://review.opendev.org/71772111:48
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Update used cirros images  https://review.opendev.org/71772511:52
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Update ansible to 2.9.6  https://review.opendev.org/68100311:54
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Update ansible to 2.8.10  https://review.opendev.org/71772811:56
*** shyamb has joined #openstack-ansible11:56
noonedeadpunkI think we have to use ansible 2.9 for U. Probably we will have to start making use of collections, as rabbitmq is now available as collections (thanks to odyssey4me)11:57
jamesdenton_wfhmornin12:02
noonedeadpunko/12:02
jrossernoonedeadpunk: i was thinking just the same for 2.912:03
jrosserwe could make the most minimal use of collections to fix rabbitmq12:03
noonedeadpunkyeah12:03
jrosserhave you tried this yet?12:04
noonedeadpunkactually no:( hoped we can backport things on time12:04
noonedeadpunkand we also will need to find the way of their usage, as I think we'll need to use everything as collections in W12:05
noonedeadpunks/W/V/12:05
jrosseri was looking at it12:05
jrosserbut not tried yet, maybe this afternoon i have a go12:05
noonedeadpunkreally not sure what we should concentrate on right now - centos 8 or collections or mitogen lol12:06
jrossermitogen is real close12:06
noonedeadpunkI feel like I should pickup smth from it12:06
jrosserit needs two runs atm12:06
jrosserit fails on a facts gathering of the lxc's12:06
jrosserthen re-run and it works12:06
noonedeadpunkhuh12:07
noonedeadpunkmsybe we can workaround it like with resetting connection after facts gathering or smth like that?12:07
*** shyam89 has joined #openstack-ansible12:11
openstackgerritJames Denton proposed openstack/openstack-ansible-os_neutron master: Add networking-generic-switch support for Ironic integration  https://review.opendev.org/71745712:14
*** shyamb has quit IRC12:15
*** DanyC has joined #openstack-ansible12:18
*** DanyC has quit IRC12:19
*** DanyC has joined #openstack-ansible12:19
*** udesale_ has joined #openstack-ansible12:28
jrossernoonedeadpunk: this is where the mitogen lxc jobs fail https://github.com/openstack/openstack-ansible-lxc_container_create/blob/605bfb83fec3edd338c4be15626630deca00612a/tasks/lxc_container_config.yml#L61-L6312:30
jrosseri didnt spot it when i was hacking locally becasue it broke a few times to it got re-run during that12:30
jrosser*so it got...12:30
*** udesale has quit IRC12:31
*** aedc has joined #openstack-ansible12:36
*** shyam89 has quit IRC12:43
jrossermnaser: have you been increasing ANSIBLE_FORKS when you do deployments?12:55
jrosseri get 15% speedup on setup_hosts.yml for an lxc aio with it set to 2012:55
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Update global-requirement-pins  https://review.opendev.org/71774813:07
*** fghaas has joined #openstack-ansible13:12
jrossernoonedeadpunk: we should also find a way to land this https://review.opendev.org/#/c/588372/13:13
noonedeadpunkoh, yeah, I saw this pne13:13
jrossereven if it's just optional flag13:14
noonedeadpunkdo you think we should just land it as is?13:14
jrosserwould take ~10mins off booting a sandbox for me right now13:14
noonedeadpunkI mean as a library instead of collection?13:14
jrosserperhaps the plugins repo should be a collection? is that how it works?13:15
noonedeadpunkI'd rather create another one where we would place stuff but generally yes13:17
noonedeadpunkInstead of changing format of that one13:17
noonedeadpunkbut yeah, we already have bunch of libraries there13:19
noonedeadpunkSo I'd say that we should at least place https://review.opendev.org/#/c/588372/15/playbooks/library/git_requirements.py to the plugins repo13:19
* noonedeadpunk picking up lxc_hosts fix13:28
jrosseroh cool, thankyou13:29
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible stable/train: ceph: run handlers after common  https://review.opendev.org/71378413:31
*** also_stingrayza is now known as stingrayza13:39
*** spatel has joined #openstack-ansible13:46
spateljamesdenton: hey13:46
spatelI am dealing with https://bugs.launchpad.net/os-vif/+bug/183725213:46
openstackLaunchpad bug 1837252 in os-vif stein "[OSSA-2019-004] Ageing time of 0 disables linuxbridge MAC learning (CVE-2019-15753)" [High,Fix committed] - Assigned to sean mooney (sean-k-mooney)13:46
spatelDo you have any update on it?13:46
mnaserjrosser: i haven't had much luck or that much more speed out of it13:47
fridtjof[m]how do I make use of tempest and the healthcheck playbooks? so far, i've found defaults/healthcheck-vars.yml, which applies default values. I assume I should set all the tempest_ prefixed variables in my user_variables.yml? Is there a reason this is not being filled out through the user_config.yml?13:48
jamesdenton_wfhspatel i thought it was resolved already?13:49
jrossermnaser: i'm wondering if what is best for the gate speed is not the same as for actual large deployments13:49
jamesdenton_wfhspatel which Stein are you on?13:49
mnaserjrosser: i've kinda just lost hope in the ability of speeding u13:49
spateli am running stein and still seeing ageing time   0.0013:49
mnaserup ansible13:49
mnaserif it's not this that bites you, its the big number of vars, if its not that, its the large number of facts that will slow down tasks13:49
spateljamesdenton: DISTRIB_RELEASE="19.0.0.0rc3.dev6"13:50
*** Blinkiz has joined #openstack-ansible13:53
*** thuydang has joined #openstack-ansible13:55
*** Blinkiz has quit IRC13:57
jamesdenton_wfhspatel which version of OSA?13:57
spateljamesdenton: i am running os-vif==1.15.113:58
spatelsean-k-mooney confirm it has been fix in 1.15.213:58
spateljamesdenton: how do i patch this?13:58
spatelis this fix part or compute nodes or controller?13:59
jamesdenton_wfheverywhere, i suspect13:59
spateldamn it..13:59
spatelso how do i fix this now :(14:00
spatelany safe or easy way?14:00
jamesdenton_wfhI would think a minor Stein upgrade would be enough.14:00
jamesdenton_wfhminor OSA bump, i mean.14:00
spateland how to do that?14:00
jamesdenton_wfhhttps://docs.openstack.org/openstack-ansible/stein/admin/upgrades/minor-upgrades.html14:01
*** Blinkiz has joined #openstack-ansible14:01
spateljamesdenton: is there a way i can do via brctl CLI?14:01
spatelbecause i am running 100s of VM already i need to fix them also14:02
spatelif i can just change ageing with command line then it will be little relief for me14:02
openstackgerritMerged openstack/openstack-ansible-galera_server master: Ensure python3 mysql client libraries are present  https://review.opendev.org/71738114:03
jamesdenton_wfhyou could try: brctl setageing <brname> <time>14:04
jamesdenton_wfhcan't guarantee it will stick14:04
spateljamesdenton: worth trying14:04
spatelif that work then i will put stuff in crontab to make it quick fix14:05
spateljamesdenton: can i upgrade specific component to fix this ?14:07
jamesdenton_wfhos-vif :D14:07
spatelyes...14:07
spatelor just nova14:07
spateli am worried to run upgrade at this point..14:07
*** mgariepy has joined #openstack-ansible14:08
jamesdenton_wfhWell, the latest Stein OSA should have a newer Neutron tag and the changes would be inherited14:08
spatelhmm14:09
jamesdenton_wfhNot sure how old your Stein deploy is, but I would have thought this would have been good for the last 5+ mos worth of Stein deploys14:09
spatelhmmm14:10
spateljamesdenton: totally with you..14:10
spateli am only worried about if it break something in production... let me think about this...14:11
spatelupgrade is always painful..14:11
*** thuydang has quit IRC14:11
*** thuydang has joined #openstack-ansible14:12
*** thuydang has quit IRC14:12
spatelI am thinking i can just upgrade nova component to fix this bug14:12
*** thuydang has joined #openstack-ansible14:12
*** this10nly has quit IRC14:13
jamesdenton_wfhyeah, i am not sure if it comes in via nova or neutron, to be honest14:13
fridtjof[m]Hey, is there documentation on tempest? Only thing i can find seems to be still related to TripleO...14:18
fridtjof[m]* using tempest with OSA14:22
*** sshnaidm is now known as sshnaidm|afk14:24
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-pip_install master: Retire pip_install role  https://review.opendev.org/71771714:27
noonedeadpunkjrosser: huh, ansible 2.9.6 almost passing in gates14:27
noonedeadpunkah, yes, it fails in upgrade job exactly the same way...14:28
*** gary_perkins has quit IRC14:30
*** gary_perkins has joined #openstack-ansible14:32
spateljamesdenton: i run some test and looking like setageing should be my fix, i am going to deploy cronjob to run every hours and if bridge has 0 ageing it will change to 30014:40
spatelThis is short term fix while i am working on upgrade plan14:40
*** sshnaidm|afk is now known as sshnaidm14:51
jamesdenton_wfhsounds good14:53
jamesdenton_wfhyou will need to iterate thru all bridges14:53
jrosserfridtjof[m]: tempest is installed as standard by OSA14:59
jrosseris there anything specific you want to know?14:59
noonedeadpunkfridtjof[m]: fwiw we do share tempest role with tripleo folks15:01
spateljamesdenton: yes i will write smart script to check every bridge created by nova15:10
*** miloa has quit IRC15:14
fridtjof[m]jrosser: I had problems running setup-openstack through a minor upgrade, so I wanted to run the healthcheck- suite of playbooks. healthcheck-openstack is failing though, because tempest prefixed variables seem to be missing, so defaults are being used that don't match my environment.15:22
fridtjof[m]ls15:22
fridtjof[m]nvm that, wrong window ^^15:22
jrosserfridtjof[m]: if you can provide specifics in a paste at paste.openstack.org then we can try and help15:25
fridtjof[m]sure!15:26
fridtjof[m]about tempest, though - i don't think it's installed. setup-openstack only includes tempest-install when either tempest_install or tempest_run are explicitly set to True.15:27
fridtjof[m]grep -r doesn't turn up either of them in my config, or the base files that I copied them from in my initial deployment15:27
noonedeadpunkso you can set these vars either in user_variables.yml or provide them with -e argument during runtime15:29
noonedeadpunkLaunch tempest isntallation is possible with playbooks/os-tempest-install.yml15:29
fridtjof[m]running that now, thank you.15:33
fridtjof[m]but it's not installed as standard then, at least with the base user_variables.yml, right? or did that change after rocky?15:34
fridtjof[m](sorry by the way for bringing up all this rocky stuff, i'm desperately trying to get to stein at least)15:35
*** ierdem has quit IRC15:37
jrosseri guess "standard" is a movable target15:37
jrossertheres what gets installed in an AIO is as close to standard as we have15:37
jrosserbut for your own deployment you can have whatever you like in the config15:38
*** gyee has joined #openstack-ansible15:38
fridtjof[m]of course. I assumed "standard" to be what the deployment guide told you to use, which is the etc/openstack_deploy directory in the repository (if that hasn't changed since rocky)15:39
*** thuydang has quit IRC15:39
fridtjof[m]interestingly enough, setup-openstack just works now15:41
fridtjof[m]it didn't hang where it did before, so I guess leaving it for a weekend resolved that. weird.15:42
openstackgerritMerged openstack/openstack-ansible-repo_server stable/train: Exclude repo_build_global_links_path from sync  https://review.opendev.org/71484115:42
*** jamesdenton_wfh has quit IRC15:43
fridtjof[m]i should really implement proper monitoring, i guess.15:45
*** rpittau is now known as rpittau|afk16:08
*** DanyC has quit IRC16:20
*** DanyC has joined #openstack-ansible16:27
*** udesale_ has quit IRC16:30
*** evrardjp has quit IRC16:36
*** evrardjp has joined #openstack-ansible16:37
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_container_create master: Fix functional tests  https://review.opendev.org/71780617:04
*** aedc has quit IRC17:08
*** thuydang has joined #openstack-ansible17:09
*** aedc has joined #openstack-ansible17:32
openstackgerritMerged openstack/openstack-ansible-repo_server master: lsyncd.lua.j2: update coding style  https://review.opendev.org/71262517:55
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Install rabbitmq from external_repo for all distros  https://review.opendev.org/71781918:04
*** miouge3625368 has quit IRC18:06
*** miouge36253686 has joined #openstack-ansible18:06
*** Miouge has quit IRC18:07
*** jonher has quit IRC18:08
*** admin0 has quit IRC18:09
*** jonher has joined #openstack-ansible18:09
*** Miouge has joined #openstack-ansible18:10
*** prometheanfire has quit IRC18:10
*** d34dh0r53 has quit IRC18:13
*** d34dh0r53 has joined #openstack-ansible18:14
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq versions  https://review.opendev.org/71782118:15
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Install rabbitmq from external_repo for all distros  https://review.opendev.org/71781918:23
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Install rabbitmq from external_repo for all distros  https://review.opendev.org/71781918:25
*** prometheanfire has joined #openstack-ansible18:28
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Bump MariaDB version  https://review.opendev.org/71783018:29
*** andrewbonney has quit IRC18:33
*** DanyC has quit IRC18:37
*** aedc has quit IRC18:40
*** fghaas has left #openstack-ansible18:49
*** aedc has joined #openstack-ansible18:54
*** kukacz has quit IRC18:55
*** kukacz has joined #openstack-ansible18:57
fridtjof[m]how do I stop tempest from trying to create a public network for me on install?18:59
fridtjof[m](with os-tempest-install)19:00
fridtjof[m](besides it also creating users, an image, a project, a private network and so on, which is also kind of undesired imo)19:01
*** jamesdenton_wfh has joined #openstack-ansible19:06
*** DanyC has joined #openstack-ansible19:07
openstackgerritMerged openstack/openstack-ansible-os_aodh stable/train: Updated from OpenStack Ansible Tests  https://review.opendev.org/69034619:12
*** ut2k3 has quit IRC19:13
*** thuydang has quit IRC19:30
*** aedc has quit IRC19:38
*** aedc has joined #openstack-ansible19:38
noonedeadpunkfridtjof[m]: iirc you should just set tempest_public_net_name to your public net, so that https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/tasks/tempest_resources.yml#L169-L185 would be jsut "ok19:42
noonedeadpunk(and other fields should be valid as well)19:42
noonedeadpunkregarding stuff creation - this is how tempest is designed to work - it creates required env based on the it's plugin tests19:43
noonedeadpunkbut it generally cleanups after itself19:43
fridtjof[m]ah, i was taking playbooks/defaults/healthcheck-vars.yml as a reference. it has slightly different names19:44
noonedeadpunkah, and ensure subnet is set as well so thet https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/tasks/tempest_resources.yml#L208-L224 would be ok as well19:44
noonedeadpunkfridtjof[m]: but maybe you have some sense and we should run tempest_resources.yml conditionally, when some var is set as true19:46
noonedeadpunkhowever it's currently not implemented.19:46
noonedeadpunkactually you can offer a patch to fix that if you want :p19:47
fridtjof[m]i can try, although i'm not that great at ansible. seems like a small enough change to implement, though19:49
fridtjof[m]ah, the variable names are slightly different on the rocky branch (tempest_public_net_physical_type instead of _name)19:51
noonedeadpunkoh, do you have fresh deployment? are you sure you want to start with rocky? it's already in the extended maintenace....19:52
*** kleini has quit IRC19:53
fridtjof[m]i know very well :/ this is not a fresh deployment, unfortunately. I wanted to add tempest so I could have some confidence about everything working before I migrate to stein19:53
noonedeadpunkoh, I see19:54
fridtjof[m]okay, I think i'm running into a more fundamental issue with tempest now. I only have one public network, which is used by instances (of course). Obviously, neutron refuses creating a new network with a physical network that's already being used, so the playbook fails anyway, even though the names and attributes are all there now19:56
*** thuydang has joined #openstack-ansible19:57
noonedeadpunkyeah, and tempest on rocky is pretty weird actually... Like I was facing almost the same thing there...19:57
fridtjof[m]i guess i'm just going to migrate to stein then tomorrow. Hope it'll be more stable :D19:59
noonedeadpunkbut eventually at this point you're missing only tempest.conf and it's workdir19:59
noonedeadpunkas tmpest should be already installed into venv19:59
fridtjof[m]ah, yup. basically "just" tempest_post_install20:00
noonedeadpunkso you can try writing your tempest.conf with existing images, users, etc taking template as a reference https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/stable/rocky/templates/tempest.conf.j220:00
fridtjof[m]yeah, i can just manually "run" what post_install does :D20:01
noonedeadpunkso yeah, for rocky it has been used like only for ci...20:01
fridtjof[m]yeah, the only mention of tempest_install i found was in the tests/ directory20:02
*** DanyC has joined #openstack-ansible20:03
fridtjof[m](I'm just going to go to stein first, though, then try the install playbook again. it's only a setup for education fortunately, so I have no service responsibilities outside of semesters)20:04
*** jamesdenton_wfh has quit IRC20:38
*** thuydang has quit IRC20:46
*** thuydang has joined #openstack-ansible20:53
*** aedc has quit IRC21:01
*** spatel has quit IRC21:49
*** aedc has joined #openstack-ansible21:54
*** rh-jelabarre has quit IRC22:11
thuydangHi All, I'm running OSA Train and have an error during "TASK [python_venv_build : Slurp up the constraints file for later re-deployment]". The error is ""file not found: /var/www/repo/os-releases/20.0.0/rally-20.0.0-constraints.txt". The file is available in infra1_utility_container but not in infra1_repo_container. Am I missing something? Thanks22:28
*** sshnaidm is now known as sshnaidm|afk22:28
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server stable/train: Updated from OpenStack Ansible Tests  https://review.opendev.org/71475023:05
*** thuydang has quit IRC23:11
*** thuydang has joined #openstack-ansible23:13
*** tosky has quit IRC23:28
*** DanyC has quit IRC23:35
*** thuydang has quit IRC23:44

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!