| *** jbadiapa has quit IRC | 00:04 | |
| *** goldyfruit___ has joined #openstack-ansible | 00:11 | |
| *** gyee has quit IRC | 00:42 | |
| *** spatel has joined #openstack-ansible | 01:24 | |
| cjloader | anyone know the scenario for ironic jobs in osa? | 01:27 |
|---|---|---|
| cjloader | jrosser: noonedeadpunk? | 01:34 |
| *** nicolasbock has quit IRC | 01:44 | |
| *** nicolasbock has joined #openstack-ansible | 01:45 | |
| *** nicolasbock has quit IRC | 01:58 | |
| openstackgerrit | pengyuesheng proposed openstack/ansible-config_template master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/673772 | 01:59 |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible master: Update inventory group/user_secrets for inspector change in Ironic https://review.opendev.org/680553 | 02:40 |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon master: Add python3 compatibility to horizon role https://review.opendev.org/681096 | 02:40 |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible master: Update inventory group/user_secrets for inspector change in Ironic https://review.opendev.org/680553 | 02:46 |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible master: Update inventory group/user_secrets for inspector change in Ironic https://review.opendev.org/680553 | 02:57 |
| *** spatel has quit IRC | 03:03 | |
| *** PrinzElvis has quit IRC | 03:39 | |
| *** PrinzElvis has joined #openstack-ansible | 03:45 | |
| *** samc-bbc has quit IRC | 03:45 | |
| *** samc-bbc has joined #openstack-ansible | 03:47 | |
| *** udesale has joined #openstack-ansible | 04:00 | |
| *** cshen has joined #openstack-ansible | 04:13 | |
| *** ianychoi_ has joined #openstack-ansible | 04:24 | |
| *** ianychoi has quit IRC | 04:27 | |
| *** dave-mccowan has quit IRC | 04:29 | |
| *** pcaruana has joined #openstack-ansible | 04:42 | |
| *** pcaruana has quit IRC | 05:12 | |
| *** redrobot has quit IRC | 05:25 | |
| *** kopecmartin|off is now known as kopecmartin | 05:46 | |
| *** rpittau|afk is now known as rpittau | 06:01 | |
| *** luksky has joined #openstack-ansible | 06:19 | |
| *** pcaruana has joined #openstack-ansible | 06:21 | |
| ChosSimbaOne | morning. | 06:33 |
| jrosser | morning | 06:34 |
| *** radeks has joined #openstack-ansible | 06:35 | |
| ChosSimbaOne | I am having some serious issues with my openstack test installation. Single infra node and single compute all on hardware this time. So the infra node tends to crash hard. No output to tty, no trances in syslog/kern.log about what happend. I cannot switch tty on the console, only thing that works is to hard reset the server. Then it come up and frezzes within 100 sec, and it normally takes tw or three reboots for it to get stable. | 06:38 |
| ChosSimbaOne | eg. it normally happes to me when using horizon e,g login or create vm or network. I am a bit clueless to what happens. I know it is not OSA related, but you might have some good idea where to start, or could point me to the right IRC channel :-) | 06:38 |
| *** southquist has joined #openstack-ansible | 06:46 | |
| noonedeadpunk | cjloader: it's generated automatically based on the role name. And it's printed to stdout https://zuul.opendev.org/t/openstack/build/31aed5f00bea49baaa270e9dc0a54c80/log/job-output.txt#2762 | 06:58 |
| noonedeadpunk | why are you asking? | 06:58 |
| jrosser | ChosSimbaOne: these things are hard to track down | 06:59 |
| jrosser | Do you have another similar hardware? Usually swapping components one by one is a systematic way of finding what’s wrong | 07:00 |
| jrosser | It’s not got a Broadcom nic by any chance has it? | 07:01 |
| ChosSimbaOne | jrosser: yeah, it really a bummer. And it is definately a HW issue just setup an additional openstack env. with same type of hw for infra node, and it crashes same way. No it is mellanox nic. | 07:05 |
| ChosSimbaOne | a network daughter board for dell servers. 25G. | 07:05 |
| ChosSimbaOne | Just talked with my hardware guy. He says that there is a bit of bios patches for the server, so fingers crossed it will fix it. | 07:06 |
| jrosser | I think mnaser had some issue with Broadcom nic on dell r640 and vxlan causing it to wedge the box up | 07:08 |
| *** trident has quit IRC | 07:08 | |
| jrosser | So it might be nic firmware rather than bios | 07:09 |
| *** ivve has joined #openstack-ansible | 07:09 | |
| *** luksky has quit IRC | 07:10 | |
| jrosser | ChosSimbaOne: fwiw I have hundreds of Mellanox nic and they’ve never caused trouble, I also have instructions from Mellanox on how update OEM nic if dell don’t provide a tool | 07:13 |
| *** trident has joined #openstack-ansible | 07:17 | |
| ChosSimbaOne | jrosser: thanks for the inputs. Will look into that if the BIOS upgrade does not fix it :-) | 07:18 |
| *** trident has quit IRC | 07:22 | |
| *** luksky has joined #openstack-ansible | 07:23 | |
| *** trident has joined #openstack-ansible | 07:31 | |
| *** radeks has quit IRC | 07:34 | |
| noonedeadpunk | evrardjp: jrosser can you kindly take a look at https://review.opendev.org/#/c/679101/ once you have some time for reviews? | 07:43 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_swift master: Update swift_rings_check script for python3 https://review.opendev.org/681396 | 07:43 |
| *** andymccr has joined #openstack-ansible | 07:45 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_swift master: [DNM] test swift / py3 https://review.opendev.org/680946 | 07:46 |
| *** gillesMo has joined #openstack-ansible | 07:50 | |
| *** luksky has quit IRC | 07:51 | |
| jrosser | noonedeadpunk: does that work outside zuul? | 07:56 |
| *** luksky has joined #openstack-ansible | 07:57 | |
| *** asettle has joined #openstack-ansible | 07:59 | |
| jrosser | i figure "./gate_check_commit.sh os_heat linters" would lint the heat role? | 07:59 |
| *** jbadiapa has joined #openstack-ansible | 08:01 | |
| trident | ChosSimbaOne: What kernel version are you running? | 08:08 |
| evrardjp | jrosser: this is why I liked the tox -e ansible-lint myself. But I am not against moving to something central for role testing | 08:09 |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 08:09 | |
| evrardjp | (as it's unlikely you will not test functionaly too, which kinda requires the integrated for most roles) | 08:10 |
| *** luksky has quit IRC | 08:21 | |
| *** radeks has joined #openstack-ansible | 08:24 | |
| ChosSimbaOne | trident: Ubuntu 16.04 with HWE kernel 4.15 | 08:27 |
| trident | ChosSimbaOne Exact kernel version. We have seen similar problems with linux-image-4.15.0-60-generic due to https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842447 | 08:28 |
| openstack | Launchpad bug 1842447 in linux (Ubuntu) "Kernel Panic with linux-image-4.15.0-60-generic when specifying nameserver in docker-compose" [Undecided,Confirmed] | 08:28 |
| *** jawad_axd has joined #openstack-ansible | 08:29 | |
| ChosSimbaOne | Okay, uname returns | 08:29 |
| ChosSimbaOne | 4.15.0-60-generic | 08:30 |
| trident | ChosSimbaOne: In our cases network nodes that freezes. Was reproducible by running the hping mentioned in the bug report on an instance. | 08:30 |
| ChosSimbaOne | so that is that exact version | 08:30 |
| trident | ChosSimbaOne: We downgraded to 4.15.0-58 which eliminated the issue. But 4.15.0-62 that fixes it as well might be out now. | 08:30 |
| ChosSimbaOne | but your machines did not write anything to syslog or console? | 08:31 |
| ChosSimbaOne | Just a dead cursor? | 08:31 |
| ChosSimbaOne | Spookey. | 08:31 |
| ChosSimbaOne | Will try and downgrade kernel and see if that adds stability | 08:31 |
| trident | ChosSimbaOne: Yeah, nothing in logs. Just a frozen console and completely unresponsive. | 08:32 |
| trident | ChosSimbaOne: So not a regular panic in our case. | 08:32 |
| ChosSimbaOne | Okay. So if it is specifc to .60 a kernel downgrade should fix it i guess. | 08:33 |
| trident | ChosSimbaOne: It did fix it for us at least. Seems to be very specific to .60. Good luck! | 08:48 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_aodh master: Allow venv python interpreter to be overridden https://review.opendev.org/681408 | 08:51 |
| kaiokmo | morning | 08:53 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Allow venv python interpreter to be overridden https://review.opendev.org/681410 | 08:55 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_gnocchi master: Allow venv python interpreter to be overridden https://review.opendev.org/681411 | 08:59 |
| trident | ChosSimbaOne: 4.15.0.62.82 which also fixes the issue is in fact out now as well. So you could go with that instead of downgrading. | 09:00 |
| jrosser | evrardjp: review here if you have a moment https://review.opendev.org/#/c/680993 | 09:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Test python3 https://review.opendev.org/680470 | 09:05 |
| *** ianychoi_ has quit IRC | 09:09 | |
| ChosSimbaOne | trident: yeah you are rigth. I have just installed the .62 kernel and will see if that fixes it. Sound very much like it is the same bug. Glad it is likely not a severe harware bug. | 09:10 |
| ChosSimbaOne | trident: thank you for the input, it has saved me many hours of debug :-) | 09:11 |
| CeeMac | morning | 09:12 |
| trident | ChosSimbaOne: No worries! :) | 09:13 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 09:15 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: [DNM] telemetry stack python3 test https://review.opendev.org/681416 | 09:16 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_nova master: Set uwsgi to be runned under the interpreter https://review.opendev.org/681418 | 09:21 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_nova master: Set uwsgi to be runned under the same interpreter https://review.opendev.org/681418 | 09:22 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_placement master: Set uwsgi to be runned under the same interpreter https://review.opendev.org/681420 | 09:24 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_glance master: Set uwsgi to be runned under the same interpreter https://review.opendev.org/681422 | 09:26 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_cinder master: Set uwsgi to be runned under the same interpreter https://review.opendev.org/681423 | 09:28 |
| noonedeadpunk | jrosser: that what I was trying to say regarding uwsgi ^ :) | 09:29 |
| jrosser | noonedeadpunk: ah ok - so thats an important thing that the service and uwsgi venv python versions match? | 09:31 |
| noonedeadpunk | not important, but if we're trying to switch things to python 3 - it can't be acomplished without that) | 09:32 |
| noonedeadpunk | It just corresponds to our target | 09:32 |
| noonedeadpunk | use python 3 for services | 09:32 |
| noonedeadpunk | since uwsgi defaults to python2 | 09:32 |
| noonedeadpunk | we can override it globably somewhere afterwards | 09:33 |
| noonedeadpunk | but, if we're trying to do this in interative way... | 09:33 |
| noonedeadpunk | ohhhhhhhhhh | 09:33 |
| noonedeadpunk | man | 09:33 |
| noonedeadpunk | yep, it doewsn't matter | 09:33 |
| noonedeadpunk | sorry | 09:33 |
| * noonedeadpunk going to abandon all of these | 09:34 | |
| noonedeadpunk | so, I can probably set uwsgi venv to python3 and it shouldn't affect anything.... | 09:39 |
| noonedeadpunk | since it's only for uwsgi daemon binary, and it's run code inside venv... | 09:40 |
| *** prometheanfire has quit IRC | 09:43 | |
| *** prometheanfire has joined #openstack-ansible | 09:44 | |
| noonedeadpunk | So we can just set python3 inside defaults once get centos python3 ready | 09:45 |
| jrosser | as far as centos test goes i was going to redo that so it uses python2 in the ansible venv | 09:47 |
| jrosser | and try to use python3 for the services | 09:47 |
| jrosser | that selinux library missing pretty much fubar ansible | 09:47 |
| noonedeadpunk | but it's missing selinux as zbr stated... | 09:47 |
| noonedeadpunk | yeah:( | 09:47 |
| noonedeadpunk | not sure that centos for 7.7 smth will change... | 09:48 |
| jrosser | uwsgi python3 global override here hopefully, certainly for test https://review.opendev.org/#/c/680470/12/inventory/group_vars/all/all.yml | 09:49 |
| noonedeadpunk | oh, that' great! | 09:49 |
| noonedeadpunk | sorry for disturbing | 09:49 |
| zbr | noonedeadpunk: unless people start to be more verbose about selinux/py3/centos7 it will NOT work. See https://bugs.centos.org/view.php?id=16389 -- if you do not comment and use alternative channels to raise awareness, they will do nothing to make ansible work with python3 on centos7. | 09:50 |
| zbr | the reality is the python3-selinux slipped through the cracks so to speak, everyone defers building an rpm for it | 09:52 |
| zbr | centos excuse is that it was not part of rhel7, so is not up to them to build it. rhel7 doesnt care as their solution is called upgrade to rhel8. | 09:53 |
| zbr | and our (redhat) announce regarding probiding a python36 for centos7 is clearly not helping ansible users at all due to the bug i raised. | 09:54 |
| noonedeadpunk | jrosser: probably we should try to run ansible under python2, while build venvs for service under python3? | 09:57 |
| noonedeadpunk | won't it do the trick? | 09:57 |
| zbr | noonedeadpunk: not really because it will force you to install python2 on machines only for the same of being able to manage them with ansible, even if the deployed code uses only python3. | 10:01 |
| zbr | that means signifiant extra footprint which could be avoided.... with a selinux rpm. | 10:01 |
| noonedeadpunk | yep, but in terms of centos python2 exists anyway | 10:01 |
| zbr | noonedeadpunk: yeah, that is what we done so far too. | 10:02 |
| noonedeadpunk | not sure that it can be easily avoided | 10:02 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_ceilometer master: Updated from OpenStack Ansible Tests https://review.opendev.org/680038 | 10:07 |
| *** cshen has quit IRC | 10:08 | |
| *** izake has joined #openstack-ansible | 10:13 | |
| zbr | noonedeadpunk: also raised https://bugzilla.redhat.com/show_bug.cgi?id=1751163 | 10:19 |
| openstack | bugzilla.redhat.com bug 1751163 in python3 "python36 is missing selinux python bindings (preventing ansible use)" [Urgent,New] - Assigned to python-maint | 10:19 |
| *** cshen has joined #openstack-ansible | 10:19 | |
| *** ianychoi has joined #openstack-ansible | 10:30 | |
| *** ianychoi has quit IRC | 10:45 | |
| *** ianychoi has joined #openstack-ansible | 10:45 | |
| *** dr_feelgood has joined #openstack-ansible | 10:57 | |
| *** ansmith_ has quit IRC | 11:05 | |
| *** sshnaidm|ruck is now known as sshnaidm|bbl | 11:20 | |
| *** udesale has quit IRC | 11:31 | |
| *** dr_feelgood has quit IRC | 11:35 | |
| *** dr_feelgood has joined #openstack-ansible | 11:39 | |
| *** nicolasbock has joined #openstack-ansible | 11:39 | |
| jrosser | noonedeadpunk: there are some hardcoded python paths for ceilometer http://codesearch.openstack.org/?q=ceilometer_lib_dir | 11:48 |
| *** izake has quit IRC | 11:48 | |
| noonedeadpunk | yeah.... it's a bit harder than horizon... | 11:51 |
| jrosser | i can do the same thing for source installs as i did in horizon, pull out the path after the venv build | 11:52 |
| noonedeadpunk | or not? | 11:52 |
| jrosser | the distro installs look more wierd though | 11:52 |
| jrosser | unless we don't care about those | 11:52 |
| noonedeadpunk | I'd say we're supposed to care about distro installs as well | 11:53 |
| noonedeadpunk | and retrieving and overriding these files is important | 11:53 |
| jrosser | oh maybe it's fine - redhat-7.yml will only ever deal with py2 openstack packages? | 11:53 |
| noonedeadpunk | suggest to hardcode python3.6 for ceilometer?) | 11:54 |
| jrosser | and debian / suse15 will always be in some py3 path for distro packages | 11:54 |
| *** luksky has joined #openstack-ansible | 11:55 | |
| noonedeadpunk | yep, that may work btw | 11:55 |
| jrosser | ok so i'll fix the source install there, and take off the centos patch for py3 depends-on becasue thats not helping right now | 11:56 |
| *** spsurya has joined #openstack-ansible | 11:58 | |
| noonedeadpunk | don't you know where to find zuul variables to use? | 11:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon master: Allow venv python interpreter to be overridden https://review.opendev.org/680993 | 12:02 |
| noonedeadpunk | found https://zuul-ci.org/docs/zuul/user/jobs.html#zuul-variables :) | 12:02 |
| *** dave-mccowan has joined #openstack-ansible | 12:04 | |
| *** ansmith has joined #openstack-ansible | 12:08 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 12:13 |
| *** goldyfruit___ has quit IRC | 12:15 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Test python3 https://review.opendev.org/680470 | 12:21 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Add python3 compatibility for source installs https://review.opendev.org/681455 | 12:24 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: [DNM] telemetry stack python3 test https://review.opendev.org/681416 | 12:25 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: [DNM] telemetry stack python3 test https://review.opendev.org/681416 | 12:26 |
| jrosser | evrardjp: thoughts? http://paste.openstack.org/show/775113/ | 12:32 |
| openstackgerrit | Merged openstack/openstack-ansible master: Use version test instead of version_compare https://review.opendev.org/681043 | 12:34 |
| *** dr_feelgood has quit IRC | 12:37 | |
| noonedeadpunk | folks, why don't we set horizon_listen_ports to like 8080 and 8443 for https://opendev.org/openstack/openstack-ansible-os_horizon/src/branch/master/defaults/main.yml#L386-L388 | 12:57 |
| noonedeadpunk | to resolve haproxy problem | 12:59 |
| noonedeadpunk | and haproxy_backend_port to 8080 | 13:00 |
| noonedeadpunk | I don't think that for containers it matters, and for baremetal it doesn't work anyway | 13:01 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/group_vars/haproxy/haproxy.yml#L203-L204 | 13:01 |
| noonedeadpunk | jrosser evrardjp mnaser can you remind me why this idea was rejected previously (if it was ofc) ^ ? | 13:02 |
| *** dr_feelgood has joined #openstack-ansible | 13:06 | |
| *** dr_feelgood has quit IRC | 13:07 | |
| *** dr_feelgood has joined #openstack-ansible | 13:07 | |
| *** cshen has quit IRC | 13:11 | |
| *** goldyfruit___ has joined #openstack-ansible | 13:29 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 13:29 |
| *** southquist has quit IRC | 13:34 | |
| *** sshnaidm|bbl is now known as sshnaidm|ruck | 13:44 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 13:45 |
| *** redrobot has joined #openstack-ansible | 13:50 | |
| *** rpittau is now known as rpittau|afk | 13:54 | |
| *** jawad_axd has quit IRC | 13:55 | |
| jrosser | noonedeadpunk: i can't think of a good reason we don't do that | 13:56 |
| jrosser | adding horizon & haproxy to the metal CI jobs feels important | 13:56 |
| noonedeadpunk | yeah, it's just too obvious and easy thing to implement for not being here:) | 13:57 |
| noonedeadpunk | So maybe I'm missing smth or from some historical prespective... | 13:58 |
| cjloader | noonedeadpunk: was trying to run scenario locally | 14:03 |
| evrardjp | jrosser: sorry I just saw that ping | 14:03 |
| evrardjp | toabctl: are you there? | 14:03 |
| evrardjp | Who is nowadays following cloud openstack Master rpm packaging for opensuse leap 15.1? | 14:04 |
| evrardjp | as you can see the message from jrosser above ./x86_64/libuv1-1.30.1-lp151.35.1.x86_64.rpm was not found | 14:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Add python3 compatibility for source installs https://review.opendev.org/681455 | 14:06 |
| evrardjp | jrosser: for context it might be useful to say which package you're trying to install. Else it's ahrd to follow | 14:06 |
| evrardjp | I cna help a little later | 14:06 |
| jrosser | oh sorry - let me dig out the error | 14:06 |
| evrardjp | toabctl: is the pro on rpm-packaging, and I saw that this is maybe a little too complex for me, so I pinged him :D | 14:07 |
| jrosser | evrardjp: https://zuul.opendev.org/t/openstack/build/1a4f9553b879434cb5ada7474e48f1d0/log/job-output.txt#10500 | 14:07 |
| jrosser | need to decide if its an infra mirror fail or a packing problem | 14:07 |
| jrosser | *packaging | 14:08 |
| jrosser | cjloader: when you run locally you just "export SCENARIO=aio_lxc" or whatever you need | 14:08 |
| jrosser | then bootstrap-aio.sh and it generates the config | 14:09 |
| cjloader | does it include the ironic role? | 14:09 |
| jrosser | i generally don't use the gate-check-commit script when doing local tests becasue there is stuff needs fiddling with most of the time in the intermediate steps | 14:09 |
| jrosser | you need the right secenario, so lets find an os_ironic test | 14:10 |
| cjloader | yes | 14:10 |
| noonedeadpunk | jrosser: but you can pass everything as a positional arguments there | 14:10 |
| *** nicolasbock has quit IRC | 14:10 | |
| noonedeadpunk | actually CI all in all launches gate-check-commit | 14:10 |
| jrosser | https://zuul.opendev.org/t/openstack/build/31aed5f00bea49baaa270e9dc0a54c80/log/job-output.txt#2764 | 14:10 |
| cjloader | that was my question jrosser | 14:11 |
| *** nicolasbock has joined #openstack-ansible | 14:11 | |
| *** nicolasbock has quit IRC | 14:11 | |
| noonedeadpunk | so you run ./scripts/gate-check-commit.sh aio_lxc_ironic deploy source | 14:11 |
| noonedeadpunk | and that's it | 14:11 |
| cjloader | ok ty | 14:12 |
| jrosser | noonedeadpunk: where does the patch to os_ironic get applied | 14:12 |
| noonedeadpunk | I override arr like https://docs.openstack.org/openstack-ansible/latest/reference/configuration/extending-osa.html#adding-new-or-overriding-roles-in-your-openstack-ansible-installation | 14:13 |
| *** nicolasbock has joined #openstack-ansible | 14:13 | |
| jrosser | and you can reference a change in zuul like that? | 14:15 |
| jrosser | zuul/gerrit | 14:15 |
| * jrosser never did it like this :) | 14:15 | |
| *** nicolasbock has quit IRC | 14:15 | |
| cjloader | i'd *like* to be able to run my patch in both OSA and os_ironic using as less manual steps as possible | 14:16 |
| cjloader | both patches | 14:17 |
| cjloader | jrosser: noonedeadpunk | 14:17 |
| *** BjoernT has joined #openstack-ansible | 14:17 | |
| *** cshen has joined #openstack-ansible | 14:20 | |
| noonedeadpunk | Hm, I kinda can't repeat my success which I had previously with that :( | 14:23 |
| jrosser | cjloader: so what i would do is clone openstack-ansible | 14:26 |
| *** electrofelix has joined #openstack-ansible | 14:26 | |
| jrosser | then you need to apply your patch to openstack-ansible, https://review.opendev.org/#/c/680553/ | 14:28 |
| jrosser | go to that in a browser and top right use the 'download' and copy the cherry-pick line, and run that in a terminal | 14:28 |
| jrosser | that will apply your patch | 14:28 |
| *** cshen has quit IRC | 14:28 | |
| jrosser | then scripts/bootstrap-ansible.sh | 14:29 |
| jrosser | let that finish, and you'll have the ironic role in /etc/ansible/roles/os_ironic | 14:29 |
| jrosser | same thing, go to gerrit, get the cherry pick line for that role and apply the patch | 14:29 |
| cjloader | my patch? | 14:29 |
| cjloader | oh ok | 14:29 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_murano master: Initial commit to os_murano role https://review.opendev.org/677830 | 14:29 |
| *** cshen has joined #openstack-ansible | 14:30 | |
| jrosser | then back to the openstack-ansible dir, export SCENARIO=whatever-you-need | 14:30 |
| jrosser | scripts/bootstrap-aio.sh | 14:30 |
| jrosser | then deploy | 14:30 |
| cjloader | sounds good | 14:30 |
| jrosser | if it's in a VM you might take a snapshot at an good point so you can redo it quickly | 14:31 |
| guilhermesp | jrosser: I did some more debugging on this yesterday https://review.opendev.org/#/c/670403/ | 14:31 |
| guilhermesp | i found something that I'm not sure if is expected in an aio deployment | 14:32 |
| guilhermesp | https://www.irccloud.com/pastebin/sPVnBv1v/ | 14:32 |
| guilhermesp | I've been comparing with our metal deployments | 14:32 |
| guilhermesp | I suppose this here is necessary to populate this var with all the nodes in the cluster https://github.com/openstack/openstack-ansible-galera_server/blob/edd360694ef8a6650f65d7949c8f64457970ba90/defaults/main.yml#L107 | 14:33 |
| guilhermesp | I can see it populated in our metal deployments | 14:33 |
| guilhermesp | then we could add the gmcast https://galeracluster.com/library/documentation/mysql-wsrep-options.html as you did in the patch | 14:34 |
| guilhermesp | our metal deployments also listens to *:4567... | 14:35 |
| jrosser | yes - i think you have gmcast override for your deploys? | 14:35 |
| guilhermesp | that's how we overriden the galera vars | 14:35 |
| guilhermesp | https://www.irccloud.com/pastebin/8MsKd6c8/ | 14:35 |
| *** alvinstarr has quit IRC | 14:35 | |
| jrosser | imho you should do what makes sense for the metal deploys that you have | 14:36 |
| jrosser | and if thats working, then we have a validated approach for multinode that we know works | 14:36 |
| jrosser | that can't be tested in an AIO | 14:37 |
| guilhermesp | yeah, that's way I was thinking or remove the wresp overrides you have added and set vars like we did above | 14:38 |
| guilhermesp | if you permit I can be editing your patch | 14:38 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 14:38 |
| *** sshnaidm|ruck is now known as sshnaidm|rover | 14:40 | |
| jrosser | guilhermesp: please do edit :) | 14:45 |
| jrosser | guilhermesp: oh well hold on, galera_wsrep_provider_options should be left empty for the user | 14:46 |
| guilhermesp | in an aio scenario? | 14:47 |
| guilhermesp | what I'd do in your patch was to override galera_wsrep_provider_options | 14:48 |
| jrosser | actually it already has defaults https://opendev.org/openstack/openstack-ansible-galera_server/src/branch/master/defaults/main.yml#L124 | 14:49 |
| jrosser | could extend those? | 14:49 |
| *** dr_feelgood has quit IRC | 14:51 | |
| guilhermesp | yeah like adding gmcast.listen_addr | 14:51 |
| cjloader | so "export SCENARIO=aio_lxc_ironic"? noonedeadpunk jrosser | 14:53 |
| *** jbadiapa has quit IRC | 14:54 | |
| *** theintern has joined #openstack-ansible | 14:56 | |
| *** prometheanfire has quit IRC | 14:57 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 14:59 |
| *** spatel has joined #openstack-ansible | 15:01 | |
| *** ArchiFleKs has joined #openstack-ansible | 15:06 | |
| spatel | I want to evacuate compute nodes because it has hardware issue, but all running vm using local-disk ( NOT shared disk ) can i do evacuate? | 15:07 |
| *** chandankumar has quit IRC | 15:11 | |
| *** chandankumar has joined #openstack-ansible | 15:12 | |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-galera_server master: Add defaults for galera server and wrep cluster bind addresses https://review.opendev.org/670403 | 15:13 |
| jrosser | cjloader: yes i think so | 15:22 |
| *** gyee has joined #openstack-ansible | 15:23 | |
| cjloader | jrosser: seems to be working | 15:24 |
| cjloader | i'm able to reproduce my integrated repo PR | 15:25 |
| cjloader | failure | 15:25 |
| cjloader | thx | 15:26 |
| spatel | is it true if instance configured for CPU pining then you can't do any kind of migration? | 15:27 |
| spatel | I am trying block migration and it failed with NUMA error | 15:28 |
| spatel | migration-precheck error | 15:28 |
| spatel | How you guys manage this kind of issue where you don't have shared storage disk? | 15:28 |
| *** macz has joined #openstack-ansible | 15:29 | |
| *** ivve has quit IRC | 15:29 | |
| *** cshen has quit IRC | 15:32 | |
| *** nicolasbock has joined #openstack-ansible | 15:35 | |
| *** theintern has quit IRC | 15:36 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer master: Updated from OpenStack Ansible Tests https://review.opendev.org/680038 | 15:36 |
| *** cshen has joined #openstack-ansible | 15:43 | |
| *** cshen has quit IRC | 15:47 | |
| *** theintern has joined #openstack-ansible | 15:48 | |
| *** Garyx has quit IRC | 15:53 | |
| *** Garyx has joined #openstack-ansible | 15:54 | |
| *** spatel has quit IRC | 16:01 | |
| cjloader | :q | 16:01 |
| cjloader | oops | 16:02 |
| *** spatel has joined #openstack-ansible | 16:02 | |
| *** spatel has quit IRC | 16:07 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-openstack_hosts master: Convert dynamic includes to static imports https://review.opendev.org/681526 | 16:13 |
| *** luksky has quit IRC | 16:21 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 16:22 |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible master: Update inventory group/user_secrets for inspector change in Ironic https://review.opendev.org/680553 | 16:29 |
| *** chandankumar is now known as raukadah | 16:31 | |
| *** NewJorg has joined #openstack-ansible | 16:32 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 16:40 |
| *** ivve has joined #openstack-ansible | 16:42 | |
| *** hwoarang has quit IRC | 16:42 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_barbican master: Allow venv python interpreter to be overridden https://review.opendev.org/681533 | 16:43 |
| *** hwoarang has joined #openstack-ansible | 16:45 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cloudkitty master: Allow venv python interpreter to be overridden https://review.opendev.org/681534 | 16:47 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_congress master: Allow venv python interpreter to be overridden https://review.opendev.org/681536 | 16:50 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_designate master: Allow venv python interpreter to be overridden https://review.opendev.org/681537 | 16:52 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_heat master: Allow venv python interpreter to be overridden https://review.opendev.org/681538 | 16:55 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: Allow venv python interpreter to be overridden https://review.opendev.org/681541 | 16:58 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_magnum master: Allow venv python interpreter to be overridden https://review.opendev.org/681542 | 17:00 |
| *** luksky has joined #openstack-ansible | 17:26 | |
| *** sshnaidm|rover is now known as sshnaidm|off | 17:28 | |
| *** spsurya has quit IRC | 17:32 | |
| *** kopecmartin is now known as kopecmartin|off | 17:38 | |
| guilhermesp | noonedeadpunk: regarding murano patch... not sure if we are missing uwsgi configs... that the murano-api status locally http://paste.openstack.org/show/775153/ | 17:44 |
| guilhermesp | the playbooks finishes with no errors, but murano api doesn't seems functional | 17:44 |
| *** spatel has joined #openstack-ansible | 17:54 | |
| *** electrofelix has quit IRC | 17:56 | |
| *** Garyx has quit IRC | 18:12 | |
| *** Garyx has joined #openstack-ansible | 18:16 | |
| *** jbadiapa has joined #openstack-ansible | 18:27 | |
| *** radeks has quit IRC | 18:40 | |
| guilhermesp | jrosser: around? | 18:43 |
| jrosser | yeah | 18:43 |
| guilhermesp | all right to | 18:43 |
| guilhermesp | I came to a conclusion about the patch to galera | 18:44 |
| guilhermesp | conparing with two of our metal deployments | 18:44 |
| guilhermesp | for a multi node metal deployment, we don't need to apply the configs here https://review.opendev.org/#/c/670403/... also I noticed locally that the patch fails because of https://review.opendev.org/#/c/670403/3/templates/my.cnf.j2 | 18:44 |
| guilhermesp | in our metal deployment, we just override variables in group_vars | 18:45 |
| guilhermesp | as I exposed earlier | 18:45 |
| guilhermesp | and I can see in each of controller node the galera ports being binded to the management address | 18:45 |
| guilhermesp | so what I'd suggest is to abandon the galera_server patch, and then I edit the group _vars in the integrated patch to point the confgs we are using in our deployments | 18:46 |
| jrosser | ok, sounds good | 18:47 |
| guilhermesp | right, let me edit the integrated patch to remove the depends on and add the group_vars | 18:47 |
| guilhermesp | then later we can abandon the galera_server patch | 18:48 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible master: [WIP] Bind services to mgmt network addresses https://review.opendev.org/670051 | 18:51 |
| guilhermesp | so as I'm editing the patches I will abandon later with a message explaining the reasons, right jrosser ? | 18:52 |
| guilhermesp | if the integrated patch passes, I think we are ok with bind-to-mgmt topic | 18:52 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_designate master: Allow venv python interpreter to be overridden https://review.opendev.org/681537 | 18:54 |
| jrosser | guilhermesp: yes thats fine | 18:55 |
| jrosser | guilhermesp: i think the only thing to do is double check for all the roles that there is an override for the mgmt address | 18:56 |
| guilhermesp | you mean in a general way to the topic right? | 18:57 |
| guilhermesp | check here https://review.opendev.org/#/c/670051/21 if we are being able to override for the mgmt address | 18:57 |
| jrosser | becasue the group vars that we set are not yet for all the groups | 18:57 |
| jrosser | so just randomly looking we don't yet place a var for zun, ceilometer, aodh | 18:59 |
| *** earnThis has joined #openstack-ansible | 19:07 | |
| earnThis | does rackspace offer support for community ansible code? | 19:09 |
| cjloader | earnThis: what do you mean? | 19:13 |
| earnThis | @cjloader: to what extent does rackspace support ansible | 19:14 |
| cjloader | OSA or ansible proper? | 19:14 |
| earnThis | both | 19:15 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Add python3 compatibility for source installs https://review.opendev.org/681455 | 19:23 |
| earnThis | cjloader: thoughts? | 19:32 |
| spatel | noonedeadpunk: hey! | 19:33 |
| spatel | I figured it out how to pull metrics from libvirt and push into dashboard | 19:33 |
| jrosser | guilhermesp: since i started all the bind-to-mgmt work the uwsgi role changes have merged | 19:40 |
| jrosser | so this makes it a whole lot easier to fix roles which don't have a default var for the bind address yet | 19:40 |
| *** CeeMac has quit IRC | 19:41 | |
| jrosser | and we can probably do it without the many group vars too, by adding a global var like i'm doing for the py3 patches | 19:43 |
| *** theintern has quit IRC | 20:01 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_manila master: Allow venv python interpreter to be overridden https://review.opendev.org/681588 | 20:03 |
| guilhermesp | oh dear still failing ? https://eeb8c6fc528890e35b82-8e40407516a8c23d7cf04400cc86fdf3.ssl.cf1.rackcdn.com/670051/21/check/openstack-ansible-deploy-aio_metal-debian-stable/ba2074e/logs/ara-report/ | 20:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_masakari master: Add python3 support for suse https://review.opendev.org/681590 | 20:07 |
| *** BjoernT has quit IRC | 20:08 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_mistral master: Allow venv python interpreter to be overridden https://review.opendev.org/681591 | 20:11 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Allow venv python interpreter to be overridden https://review.opendev.org/681592 | 20:14 |
| *** cshen has joined #openstack-ansible | 20:15 | |
| jrosser | guilhermesp: there are ACL in galera i think? maybe it used to be localhost / 127.0.0.1? | 20:16 |
| jrosser | hmm now whats going on with the os_manila role - there dont seem to be any useful tests running | 20:17 |
| *** spatel has quit IRC | 20:18 | |
| *** cshen has quit IRC | 20:20 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_panko master: Allow venv python interpreter to be overridden https://review.opendev.org/681593 | 20:20 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_rally master: Allow venv python interpreter to be overridden https://review.opendev.org/681595 | 20:24 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_sahara master: Allow venv python interpreter to be overridden https://review.opendev.org/681596 | 20:27 |
| *** markvoelker has quit IRC | 20:27 | |
| *** ansmith has quit IRC | 20:28 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tacker master: Allow venv python interpreter to be overridden https://review.opendev.org/681598 | 20:29 |
| *** markvoelker has joined #openstack-ansible | 20:36 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tempest master: Allow venv python interpreter to be overridden https://review.opendev.org/681600 | 20:37 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_trove master: Allow venv python interpreter to be overridden https://review.opendev.org/681602 | 20:39 |
| *** markvoelker has quit IRC | 20:41 | |
| *** BjoernT has joined #openstack-ansible | 20:47 | |
| *** BjoernT has quit IRC | 20:51 | |
| *** BjoernT_ has joined #openstack-ansible | 20:51 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Use python3 source installs of OpenStack services where possible https://review.opendev.org/680470 | 20:53 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_zun master: Allow venv python interpreter to be overridden https://review.opendev.org/681608 | 21:03 |
| *** ivve has quit IRC | 21:05 | |
| *** ivve has joined #openstack-ansible | 21:05 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Update ansible to 2.8.4 https://review.opendev.org/681610 | 21:05 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/stein: Update ansible to 2.7.13 https://review.opendev.org/681611 | 21:07 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/rocky: Update ansible to 2.5.14 https://review.opendev.org/681612 | 21:09 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Ensure python3-systemd is present on all hosts for suse systems https://review.opendev.org/681183 | 21:21 |
| *** prometheanfire has joined #openstack-ansible | 21:26 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq & erlang versions https://review.opendev.org/681614 | 21:26 |
| guilhermesp | not sure jrosser .... this task is passing locally. I might ask a hold for infra to dig in a little more | 21:29 |
| *** goldyfruit_ has joined #openstack-ansible | 21:32 | |
| *** goldyfruit___ has quit IRC | 21:34 | |
| *** gillesMo has quit IRC | 21:46 | |
| *** markvoelker has joined #openstack-ansible | 21:59 | |
| donnyd | this is probably outside the openstack-ansible wheelhouse, but I am hoping someone has done something similar in here | 22:01 |
| donnyd | I need to change the url for glance to include a prefix so something like http://ip_address:9292/image/v2 | 22:01 |
| donnyd | Would anyone know how I might go about something like that | 22:02 |
| jrosser | donnyd: might start here? https://www.haproxy.com/documentation/aloha/10-0/traffic-management/lb-layer7/http-rewrite/#rewriting-http-urls | 22:04 |
| donnyd | jrosser: I was afraid of that | 22:05 |
| donnyd | i use an acl to determine services by path already, and i didn't really want to have to use haproxy to strip that part of the request off | 22:06 |
| donnyd | but it may be my only option... the rest of the services are too easy | 22:07 |
| donnyd | just glance | 22:07 |
| donnyd | thanks :) | 22:07 |
| *** nicolasbock has quit IRC | 22:08 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Allow venv python interpreter to be overridden https://review.opendev.org/681592 | 22:20 |
| *** nicolasbock has joined #openstack-ansible | 22:26 | |
| jrosser | tons to review/merge here https://review.opendev.org/#/q/topic:osa-py3+(status:open+OR+status:merged) | 22:32 |
| *** threestrands has joined #openstack-ansible | 22:37 | |
| *** luksky has quit IRC | 22:38 | |
| *** mathlin has quit IRC | 22:54 | |
| *** mathlin has joined #openstack-ansible | 22:54 | |
| *** mathlin has quit IRC | 22:55 | |
| *** brad[] has quit IRC | 22:57 | |
| *** goldyfruit_ has quit IRC | 22:57 | |
| *** brad[] has joined #openstack-ansible | 22:58 | |
| *** mathlin has joined #openstack-ansible | 22:59 | |
| *** mathlin has quit IRC | 23:00 | |
| *** macz has quit IRC | 23:01 | |
| *** mathlin has joined #openstack-ansible | 23:04 | |
| *** BjoernT_ has quit IRC | 23:18 | |
| *** devx has quit IRC | 23:27 | |
| *** devx has joined #openstack-ansible | 23:29 | |
| *** BjoernT has joined #openstack-ansible | 23:39 | |
| *** BjoernT has quit IRC | 23:48 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!