| *** CeeMac has quit IRC | 00:24 | |
| *** ianychoi has joined #openstack-ansible | 00:46 | |
| *** djhankb has quit IRC | 01:06 | |
| *** djhankb has joined #openstack-ansible | 01:06 | |
| *** kleini_ has quit IRC | 01:55 | |
| *** Anticimex has quit IRC | 01:56 | |
| *** Anticimex has joined #openstack-ansible | 01:59 | |
| *** rh-jelabarre has quit IRC | 02:32 | |
| *** evrardjp has quit IRC | 02:33 | |
| *** evrardjp has joined #openstack-ansible | 02:33 | |
| *** snapdeal has joined #openstack-ansible | 04:16 | |
| *** NewJorg has quit IRC | 05:04 | |
| *** NewJorg has joined #openstack-ansible | 05:08 | |
| *** miloa has joined #openstack-ansible | 05:34 | |
| *** sshnaidm|pto has quit IRC | 05:47 | |
| *** sshnaidm|pto has joined #openstack-ansible | 05:48 | |
| *** miloa has quit IRC | 05:57 | |
| *** gyee has quit IRC | 06:59 | |
| *** andrewbonney has joined #openstack-ansible | 07:24 | |
| *** luksky has joined #openstack-ansible | 07:34 | |
| *** rpittau|afk is now known as rpittau | 07:45 | |
| *** tosky has joined #openstack-ansible | 07:46 | |
| *** luksky has quit IRC | 08:31 | |
| *** luksky has joined #openstack-ansible | 08:31 | |
| *** luksky has quit IRC | 08:42 | |
| *** luksky has joined #openstack-ansible | 08:55 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Optionally build container image as a local chroot https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/776201 | 09:33 |
|---|---|---|
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Switch all base images to be built locally rather than downloaded https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/778713 | 09:33 |
| *** dpawlik9 has quit IRC | 09:46 | |
| *** snapdeal has quit IRC | 09:52 | |
| openstackgerrit | Andrew Bonney proposed openstack/openstack-ansible-galera_server master: Listen on an additional port for monitoring/diagnostic purposes https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/786381 | 09:53 |
| *** dpawlik4 has joined #openstack-ansible | 09:59 | |
| *** djhankb has quit IRC | 10:29 | |
| *** pto has quit IRC | 10:59 | |
| *** pto has joined #openstack-ansible | 11:00 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Optionally build container image as a local chroot https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/776201 | 11:06 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Switch all base images to be built locally rather than downloaded https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/778713 | 11:06 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Update cells data if config is changed https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/786389 | 11:10 |
| *** shyamb has joined #openstack-ansible | 11:27 | |
| *** shyam89 has joined #openstack-ansible | 11:27 | |
| *** shyam89 has quit IRC | 11:43 | |
| *** shyamb has quit IRC | 11:43 | |
| *** pto has quit IRC | 11:48 | |
| *** pto has joined #openstack-ansible | 11:48 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Use template URLs in Cell Mappings https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/786258 | 11:51 |
| *** rh-jelabarre has joined #openstack-ansible | 12:03 | |
| *** CeeMac has joined #openstack-ansible | 12:05 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Optionally build container image as a local chroot https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/776201 | 12:06 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: [goal] Deprecate the JSON formatted policy file https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/782286 | 12:09 |
| jrosser | oh dear https://github.com/openstack/openstack-ansible-openstack_hosts/commit/e55c4e3defc208278759709a5fb8f7e141b485c6#diff-3d0ff1709ca48add100327bb2a468e6c508fb92a159c64c4f99ad1df89d9bddeR81 | 12:10 |
| jrosser | dynamic inventory doesnt count for functional tests, so is_metal doesnt exist there | 12:10 |
| noonedeadpunk | doh...... | 12:11 |
| jrosser | things looking totally red on my lxc_hosts patch from that | 12:11 |
| noonedeadpunk | well... all functional are lxc right? | 12:12 |
| noonedeadpunk | so we can add to test-vars.yml is_metal: false? | 12:12 |
| noonedeadpunk | Nah, then we wont configure CI VM properly :( | 12:13 |
| noonedeadpunk | oh, we can set is_metal: "{{ (ansible_facts['virtualization_type'] not in ['lxc', 'systemd-nspawn']) }}" ?:) | 12:14 |
| noonedeadpunk | let me check.... | 12:14 |
| *** shyamb has joined #openstack-ansible | 12:14 | |
| noonedeadpunk | I mean I'd rather define this var for functional tests rather then not trust dynamic inventory | 12:15 |
| jrosser | so long as we gather the virt type facts | 12:16 |
| noonedeadpunk | then we will write some hook for functional tests..... Don't think we're super close to drop them everywhere anyway | 12:17 |
| noonedeadpunk | (but eventually pretty close) | 12:18 |
| noonedeadpunk | just keystone, lxc and I guess smth minor left | 12:18 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-tests master: Add is_metal variable https://review.opendev.org/c/openstack/openstack-ansible-tests/+/786396 | 12:22 |
| *** shyamb has quit IRC | 12:25 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Use template URLs in Cell Mappings https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/786258 | 12:30 |
| *** snapdeal has joined #openstack-ansible | 12:37 | |
| *** snapdeal has quit IRC | 12:42 | |
| *** snapdeal has joined #openstack-ansible | 12:44 | |
| *** shyamb has joined #openstack-ansible | 12:50 | |
| *** spatel_ has joined #openstack-ansible | 13:01 | |
| *** spatel_ is now known as spatel | 13:01 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Switch all base images to be built locally rather than downloaded https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/778713 | 13:25 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Add centos-8 stream jobs https://review.opendev.org/c/openstack/openstack-ansible/+/776226 | 13:29 |
| jrosser | noonedeadpunk: btw i may have found the slowness on stream, i'm trying disable/mask the systemd-logind service | 13:29 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Add centos-8 stream jobs https://review.opendev.org/c/openstack/openstack-ansible/+/776226 | 13:33 |
| jrosser | ah great, looks like its 2x faster now on here for centos https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/776201 | 13:34 |
| noonedeadpunk | oh, that's interesting! | 13:37 |
| *** shyam89 has joined #openstack-ansible | 13:38 | |
| noonedeadpunk | oh, so it's just normal time now! | 13:38 |
| *** shyam89 has quit IRC | 13:38 | |
| jrosser | yup | 13:38 |
| noonedeadpunk | were some verifications with logind? Like the ones with gssapi for ssh that could slow you down? | 13:39 |
| *** shyamb has quit IRC | 13:39 | |
| jrosser | there was loads of noise in the journal from logind on the dnf chroot AIO, not errors or anything, just one entry per ansible task run | 13:39 |
| jrosser | then in the docker rootfs one instead of that there was an error instead, but checking the status of logind on those containers they are all masked, I guess setup that way in the base image already when we download it | 13:40 |
| jrosser | it's really odd actually watching what happens, it's like a few of the ansible tasks just hang for really quite a long time | 13:41 |
| jrosser | then if i take the exact same task and put it in a test.yml to run the same thing 100x over it just runs straight through with no problem at all | 13:42 |
| *** spatel has quit IRC | 13:45 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Use template URLs in Cell Mappings https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/786258 | 13:57 |
| noonedeadpunk | that exactly what I saw in the image from lxcontainers... | 14:02 |
| noonedeadpunk | that's awesome that you've figured out this, as I would never thought about logind | 14:03 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Install spice-html5 from source for RedHat https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/786447 | 14:17 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Set DynamicUser in systemd-journal-upload unit file https://review.opendev.org/c/openstack/openstack-ansible/+/630671 | 14:27 |
| *** snapdeal has quit IRC | 14:44 | |
| admin0 | in what places/jobs the admin password is use ... ( one team member left the company ) and we changed the admin pass .. i know its used in the util containers and I have updated the secrets file | 14:59 |
| admin0 | is there any playbook i need to run .. | 14:59 |
| admin0 | so far manually updated the openrc in the util containers and updated the pass in the secrets file | 15:00 |
| noonedeadpunk | admin0: well, eventually, you kind of need to rotate all service passwords as well. As they all kind of have admin access as well | 15:03 |
| noonedeadpunk | jrosser: returning back to galera issue you saw. How much active threads do you have? As I actually decided to check one of V envs and found 1400 Threads_connected, which is kind of insane on pretty calm cluster | 15:08 |
| admin0 | noonedeadpunk, right . and is is just re-running setup-openstack again ? | 15:10 |
| noonedeadpunk | I guess that's actually related to https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/master/defaults/main.yml#L94 | 15:10 |
| noonedeadpunk | admin0: well... yes, but this will cause some downtime | 15:10 |
| noonedeadpunk | as passowrd will be changed before restarting service | 15:11 |
| noonedeadpunk | and it will be changed at once even when you're running against all hosts | 15:11 |
| noonedeadpunk | *single host | 15:11 |
| jrosser | andrewbonney: ^^^ you’ve been looking at these galera issues in detail | 15:15 |
| andrewbonney | Hi. Yes, we've had threads_connected hovering around 1000 | 15:16 |
| andrewbonney | I came to the same conclusion around max_pool_size in various components | 15:16 |
| andrewbonney | Which doesn't match well with the way the default max_connection for galera is calculated | 15:16 |
| andrewbonney | I have an unconfirmed suspicion that in certain keepalived failover scenarios the 'steady state' number of connections could actually double, which might be why we recently saw an issue | 15:17 |
| noonedeadpunk | But for example I see plenty of heat connects as well, and we use there default pool size which is 5... | 15:18 |
| andrewbonney | https://codesearch.opendev.org/?q=_db_max_pool_size%7C_db_pool_size&i=nope&files=&excludeFiles=&repos= | 15:18 |
| noonedeadpunk | And it's nowhere close to 5 connections | 15:18 |
| andrewbonney | Does that take account of how many instances you have in the HA set? | 15:18 |
| noonedeadpunk | well, for heat I have 96 connections. | 15:21 |
| jrosser | are we being unlucky that in one place we have nova/neutron/ocatvia/zun all of which have that set to 120 | 15:21 |
| noonedeadpunk | 3 containers, default max_pool_size supposed to be 5 | 15:21 |
| andrewbonney | Interesting. I haven't looked at heat. Just spotted nova/neutron but the cause seemed obvious given the defaults | 15:22 |
| noonedeadpunk | well, yes. for nova I have 500, which is super insane | 15:22 |
| noonedeadpunk | like half of all used ones | 15:23 |
| andrewbonney | jrosser: we see the same in our instance which doesn't run octavia/zun | 15:23 |
| noonedeadpunk | I think if we should define max_pool_size variable for the rest of the services... as might as it's not working properly unles explicitly defined... | 15:24 |
| andrewbonney | Makes sense, and presumably a lower default as I can't see 120 (per instance) being helpful in most cases | 15:25 |
| noonedeadpunk | eah | 15:25 |
| noonedeadpunk | oh, well... For heat I thin keach service is capped to 5. So having heat-api, heat-engine and heat-api-cfn might give us 5*3 connections. Still doesn't explain 96 though... | 15:27 |
| noonedeadpunk | and playing with max_pool_size has no effect | 15:31 |
| andrewbonney | Got to disappear shortly but I can pick this up and investigate further tomorrow if needed | 15:34 |
| noonedeadpunk | andrewbonney: yeah, I think I need to went out from laptop as well... Will try to look into this as well | 15:35 |
| jrosser | well https://review.opendev.org/776226 is looking like pretty much all passing (apart from the usual focal/maridb problem) | 15:38 |
| noonedeadpunk | btw, https://review.opendev.org/c/openstack/openstack-ansible/+/785800 is passing upgrade now | 15:39 |
| noonedeadpunk | but I'm not sure about https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/786258 tbh, because of the bug nova has in nova-mange. We kind of workaround it, but dunno... | 15:40 |
| noonedeadpunk | and it feels like backporting might be a good idea, as at the moment we just leave nova in unusable state if let's say password or IP is changed for galera/rabbit | 15:41 |
| noonedeadpunk | and manual cell update is required | 15:41 |
| jrosser | hmm yes i saw the chat in the nova channel | 15:43 |
| jrosser | looks like a tricky mess there | 15:43 |
| noonedeadpunk | https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/786389 can be done as alternative, but it's "validly" failed for upgrade I guess... | 15:44 |
| noonedeadpunk | well, not really tricky... but a mess | 15:45 |
| noonedeadpunk | and we kind of supprot it | 15:45 |
| noonedeadpunk | (in terms of adding map_instances till now) | 15:46 |
| jrosser | hmm so really now the two big blockers are focal/galera and the rabbitmq ssl | 15:52 |
| jrosser | plus merge-all-the-things | 15:53 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Switch all base images to be built locally rather than downloaded https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/778713 | 15:57 |
| *** rh-jelabarre has quit IRC | 15:57 | |
| *** gyee has joined #openstack-ansible | 15:57 | |
| *** spatel_ has joined #openstack-ansible | 16:06 | |
| *** spatel_ is now known as spatel | 16:06 | |
| *** rpittau is now known as rpittau|afk | 16:09 | |
| noonedeadpunk | we have so much to merge.... | 16:10 |
| jrosser | i was just looking down the failing jobs, kind of so much there that i've lost track of all the things that need fixing | 16:13 |
| jrosser | need to get a bit more organised about this to work through it | 16:14 |
| noonedeadpunk | Well I know that bunch of things need master bump to land... | 16:16 |
| *** rh-jelabarre has joined #openstack-ansible | 16:16 | |
| noonedeadpunk | iirc magnum and octavia patcches | 16:16 |
| *** rh-jelabarre has quit IRC | 16:16 | |
| *** rh-jelabarre has joined #openstack-ansible | 16:17 | |
| jrosser | noonedeadpunk: did you just fix this with a different patch https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/425845 | 16:21 |
| jrosser | or was that the nova equialent of the same thing | 16:21 |
| noonedeadpunk | I had the one for nova only | 16:21 |
| noonedeadpunk | But we were facing an issue only when these volumes were in use | 16:22 |
| noonedeadpunk | as I can think of this happening only when some amount of volumes are being created from the image | 16:23 |
| jrosser | right yes thats what i was thinking | 16:23 |
| noonedeadpunk | for us it was qemu which was kicking the limit | 16:24 |
| noonedeadpunk | I'd say that's valid patch, except we have now better way of doing that | 16:24 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Implement WEBSSO redirect https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/786467 | 16:31 |
| jrosser | jmccrory: you around? | 16:32 |
| *** spatel has quit IRC | 16:39 | |
| *** spatel_ has joined #openstack-ansible | 16:45 | |
| *** spatel_ is now known as spatel | 16:45 | |
| jrosser | fungi: do you log user-agent for gitea traffic? | 17:25 |
| fungi | jrosser: yes | 17:25 |
| *** zul has joined #openstack-ansible | 17:25 | |
| jrosser | i'm just wondering if adjusting git config http.useragent in OSA deployments might be a worthwhile thing to do | 17:25 |
| jrosser | so that it's specifically identifiable | 17:26 |
| fungi | that might be hard if you're using command-line git (which i think the ansible git module calls out to?) | 17:26 |
| fungi | the requests we were seeing all had "git/2.7.4" as the user agent | 17:26 |
| jrosser | it's all really via pip in this case | 17:26 |
| jrosser | pip install <sha-from-repo> | 17:27 |
| jrosser | so if git called by pip will respect a global git config on the user-agent that may be helpful | 17:27 |
| fungi | maybe. it's certainly worth testing | 17:30 |
| jrosser | i can try that quite straightforwardly i think | 17:31 |
| jrosser | fungi: if i try to clone something what do you need to look in the log? the source IP? | 17:37 |
| fungi | yeah, i can track it down with that | 17:43 |
| fungi | it's unfortunately a bit indirect to map that up through the proxying we do, but not hard for a single client | 17:43 |
| jrosser | its 132.185.116.134 | 17:44 |
| fungi | thanks, looking | 17:45 |
| jrosser | hrrm | 17:46 |
| fungi | request was at 17:45:23 and handed off to gitea02 via source port 51256 | 17:46 |
| fungi | now to just find that in the apache log | 17:46 |
| jrosser | i fumbled it a bit and just now successfully cloned openstack-ansible | 17:47 |
| fungi | 38.108.68.124:51256 - - [15/Apr/2021:17:45:22 +0000] "GET /openstack-ansible-config_template/info/refs?service=git-upload-pack HTTP/1.1" 404 6305 "-" "jrosser/test" | 17:47 |
| fungi | that was a 404, but it came through with your ua | 17:47 |
| jrosser | yeah i fumbled the URL and got a "enter password..." prompt | 17:47 |
| jrosser | so thats just with git clone at the CLI on this | 17:48 |
| fungi | request at 17:46:36 used source port 54982 | 17:48 |
| jrosser | not i'll make it do something with git via pip | 17:48 |
| jrosser | *now | 17:48 |
| fungi | 38.108.68.124:54982 - - [15/Apr/2021:17:46:37 +0000] "POST /openstack/openstack-ansible.git/git-upload-pack HTTP/1.1" 200 21008441 "-" "jrosser/test" | 17:48 |
| fungi | lgtm | 17:48 |
| *** andrewbonney has quit IRC | 17:50 | |
| jrosser | fungi: hopefully you'll find something cloning keystone from the same IP just now | 17:53 |
| fungi | lookin... | 17:54 |
| fungi | 38.108.68.124:44340 - - [15/Apr/2021:17:52:36 +0000] "GET /openstack/keystone/info/refs?service=git-upload-pack HTTP/1.1" 200 3891888 "-" "jrosser/test" | 17:55 |
| fungi | bingo! | 17:55 |
| jrosser | awesome! | 17:55 |
| jrosser | well tbh thats a trivial patch, iirc theres an ansible module to futz with git config | 17:55 |
| fungi | this seems like it could be a huge help | 17:55 |
| spatel | noonedeadpunk what is the plan for 2.1.2 release? | 17:57 |
| spatel | noonedeadpunk what is the plan for 22.1.2 release? | 17:58 |
| jrosser | spatel: the SHA bump is merged here https://review.opendev.org/c/openstack/openstack-ansible/+/785803 | 18:09 |
| jrosser | just not tagged yet i think | 18:09 |
| jrosser | so thats the commit ID it will be | 18:09 |
| spatel | Thank you! | 18:10 |
| *** ChiTo has joined #openstack-ansible | 18:36 | |
| ChiTo | Hi OSA team | 18:37 |
| ChiTo | I am installing a new deployment with Train, but it looks like the gnocchi component is not building the -constraints.txt, the error I got in python_wheel_build.log is a syntax error, is there any way to set a specific version for pip on the user_variables, not sure if has to do with pip: | 18:38 |
| spatel | ChiTo Hi | 18:38 |
| ChiTo | http://paste.openstack.org/show/804522/ | 18:39 |
| ChiTo | and the python_wheel_build.log says: | 18:39 |
| ChiTo | http://paste.openstack.org/show/804523/ | 18:39 |
| ChiTo | hi spatel | 18:39 |
| ChiTo | perhaps since I am using an "old" version the master for gnocchi has changed and maybe that is why I am getting the error | 18:40 |
| spatel | hmm try -e rebuild_venv=yes if that help | 18:41 |
| ChiTo | thx spatel, let me try with the rebuild_venv | 18:42 |
| ChiTo | spatel: unfortunately the error was raised again | 18:46 |
| ChiTo | for some reason the /var/www/repo/os-releases/20.1.5/gnocchi-20.1.5-constraints.txt is not created | 18:46 |
| spatel | yes because of that error in build log | 18:47 |
| ChiTo | yes, the syntax error is struggling with that specific version | 18:47 |
| spatel | curious why are you building train? | 18:47 |
| spatel | i would say you should be using Ussuri or Victoria | 18:48 |
| ChiTo | because I wanted to replicate an existing environment with the same version | 18:48 |
| ChiTo | on my first site is Train and I would like to match the 2nd site and then to upgrade them eventually | 18:48 |
| ChiTo | every openstack component worked except for gnocchi | 18:49 |
| ChiTo | in terms of the constraints files | 18:49 |
| spatel | hmm | 18:49 |
| ChiTo | and unfortunately I have centos 7.x only :( | 18:50 |
| spatel | I am also running 7.x and now working on to move everything to ubuntu | 18:50 |
| ChiTo | me too :( | 18:52 |
| ChiTo | but at least I need to keep this environment 6-8 months more on CentOS 7.x | 18:52 |
| spatel | also check build logs on repo and see if you can find some clue.. | 18:54 |
| ChiTo | spatel: agreed, I think the only error raised is the syntax error for setuptools on the utils.py file | 18:56 |
| jrosser | ChiTo: do you actually need the gnocchi component installed, i.e you are running ceilometer? | 19:04 |
| jrosser | spatel: remember there are "bad things" happen with rebuild_venv=yes | 19:05 |
| spatel | jrosser yes but looking at ChiTo error its total unrelated | 19:05 |
| spatel | SyntaxError: invalid syntax, look like some python setuptool related issue | 19:08 |
| jrosser | ChiTo: you can maybe try moving the version here to 44.1.0 https://github.com/openstack/openstack-ansible/blob/stable/train/global-requirement-pins.txt#L16 | 19:14 |
| jrosser | thats the last version of setuptools that supports python2 | 19:15 |
| openstackgerrit | Merged openstack/openstack-ansible master: Test dashboard only for horizon scenario https://review.opendev.org/c/openstack/openstack-ansible/+/782376 | 19:16 |
| openstackgerrit | Merged openstack/openstack-ansible master: Set buster jobs to NV https://review.opendev.org/c/openstack/openstack-ansible/+/782681 | 19:17 |
| spatel | jrosser that is good catch!! | 19:19 |
| jrosser | well taking a guess really :) | 19:21 |
| ChiTo | jrosser: that is correct, I will run ceilometer too, I will try the setuptools latest version for python 2 | 19:21 |
| ChiTo | thanks for the hint | 19:21 |
| ChiTo | 44.1.0? or 41.4.0? | 19:22 |
| ChiTo | if I modify manually the global-requirements it should not be overwritten right? | 19:22 |
| jrosser | 44.1.0 I think | 19:22 |
| jrosser | only if you switch branch or checkout some new version of the code | 19:23 |
| ChiTo | understood, then I need to leave it as master from the github repo right? | 19:23 |
| jrosser | errr no | 19:23 |
| jrosser | I’m saying to leave your repo on whatever point you have on Train branch | 19:24 |
| jrosser | but just for the purpose of seeing if the setuptools version needs updating, to just modify the global pins file and try rebuilding the gnocchi venv | 19:25 |
| ChiTo | oh I see, but my assumption is that setting the "Train" branch is automatically grabbed since I am using OSA 20.1.5? | 19:27 |
| ChiTo | or how can I know what specific version of gnocchi should match the train version? | 19:27 |
| jrosser | you’ve checked out a tag, right? | 19:28 |
| ChiTo | yep but when i bootstrap I just cloned the 20.1.5 | 19:28 |
| ChiTo | no tags specifically | 19:29 |
| jrosser | well 20.1.5 is a tag | 19:30 |
| jrosser | it’s just a label on a branch, in this case the stable/train branch | 19:30 |
| ChiTo | oh got it sorry for the confusion | 19:30 |
| jrosser | so I’m not saying to change any of that at all | 19:31 |
| ChiTo | agreed | 19:31 |
| ChiTo | I am leaving it with 20.1.5 | 19:31 |
| jrosser | just edit the global pins file as you have it now, and update the setuptools version | 19:31 |
| ChiTo | but then I will place the global-requirements-pins | 19:31 |
| ChiTo | is the global-requirements-pins located in /etc/openstack-deploy? | 19:32 |
| jrosser | ? not, just edit the file you have and change the setuptools version to 44.1.0 | 19:32 |
| jrosser | no, it’s in the openstack-ansible directory | 19:32 |
| ChiTo | oh I see it now, thanks jrosser, let me try with the 44.1.0 thx! | 19:33 |
| ChiTo | redeploying now | 19:36 |
| *** dave-mccowan has quit IRC | 19:43 | |
| *** macz_ has joined #openstack-ansible | 19:55 | |
| spatel | ChiTo did that work? | 20:10 |
| ChiTo | spatel, jrosser: Hi guys, it did work!, but now I am getting an issue with the DB sync: | 20:13 |
| spatel | why don't you drop db and re-run setup-host and all related playbook | 20:13 |
| ChiTo | http://paste.openstack.org/show/804527 | 20:15 |
| ChiTo | it looks like the revision is not found but from the constraints file it was listed the 4.3.5.dev1 | 20:16 |
| spatel | i would say drop gnochi DB and try | 20:17 |
| ChiTo | I agree let me change to 4.3.5 without dev1 and retry it and then just dropping the db | 20:21 |
| jrosser | that’s the db version revision, not the gnocchi package version | 20:23 |
| jrosser | the built wheel is version 4.3.5dev1 | 20:24 |
| ChiTo | I set 4.3.5 and it raised an Escalation succeeded error from Ansible, now I rollback to 4.3.5dev1 in the constraints txt | 20:29 |
| ChiTo | and now I am dropping the DB and re execute with dev1 | 20:30 |
| ChiTo | it recreated the db but I noticed the issue is related to a missing interface trying to connect to ceph | 20:41 |
| ChiTo | due the backend storage configured is through rados | 20:41 |
| ChiTo | thank you very much guys for the assistance everything should work now | 20:41 |
| ChiTo | with the setuptools newer version it worked perfectly | 20:42 |
| spatel | +1 | 20:45 |
| *** jbadiapa has quit IRC | 21:02 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/train: Bump setuptools pin to 44.1.0 https://review.opendev.org/c/openstack/openstack-ansible/+/786508 | 21:16 |
| jrosser | ChiTo: ^ hopefully we can merge that for the next Train point release | 21:16 |
| ChiTo | jrosser: thanks a lot for backporting it | 21:35 |
| ChiTo | I noticed the latest version for 2. is 44.1.1 | 21:35 |
| ChiTo | by chance have you experienced this issue? | 21:36 |
| ChiTo | http://paste.openstack.org/show/804528 | 21:36 |
| ChiTo | it is on panko but it looks like the cron module for ansible is looking for a specialtime parameter instead of special_time | 21:36 |
| jrosser | ah yes you are right it's 44.1.1, let me fix that | 21:36 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/train: Bump setuptools pin to 44.1.1 https://review.opendev.org/c/openstack/openstack-ansible/+/786508 | 21:37 |
| jrosser | ChiTo: you're right that looks like a bug in the panko role | 21:40 |
| jrosser | i wonder how that has ever worked | 21:40 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_panko master: Fix cron module special_time parameter. https://review.opendev.org/c/openstack/openstack-ansible-os_panko/+/786510 | 21:43 |
| ChiTo | I am redeploying with special_time now, thanks for proposing the merge | 21:44 |
| jrosser | thats alos broken in master so will need to merge / backport in sequence down all the branches | 21:44 |
| jrosser | that will take some time to do | 21:44 |
| ChiTo | sure, thanks. Is there any EOL considered so far for keeping maintenance to the Train branch? | 21:47 |
| spatel | jrosser i am doing minor upgrade on Victoria and all playbooks ran fine except horizon, i got this error - http://paste.openstack.org/show/804530/ | 21:47 |
| spatel | jrosser i believe you did some work on barbican-ui, is that related ? | 21:47 |
| jrosser | right, so look carefully at the error | 21:49 |
| jrosser | "designate_dashboard", "git+https://opendev.org/openstack/barbican-ui@3a78a743426376250f43508d4281e86c75067271#egg=barbican-ui", "heat_dashboard" | 21:49 |
| jrosser | see that designate and heat just give the name of the python package to install | 21:50 |
| spatel | hmm | 21:50 |
| jrosser | but for some reason barbican is wanting the source url | 21:50 |
| spatel | i didn't get it why its looking for designate_dashboard ? | 21:52 |
| spatel | I didn't tell to install | 21:53 |
| jrosser | do you have designate? | 21:54 |
| spatel | yes | 21:54 |
| jrosser | ok, so it's automatic https://github.com/openstack/openstack-ansible-os_horizon/blob/master/defaults/main.yml#L297 | 21:56 |
| spatel | hmm so i have to say horizon_enable_octavia_ui: False if i don't want.. | 21:56 |
| spatel | but curious why does it failing it was working before i am just upgrading stuff.. | 21:57 |
| jrosser | normally the url to the git repo is only ever used during the wheel build | 21:57 |
| jrosser | you should never see it when installing packages into the venv | 21:58 |
| jrosser | so i think maybe look at the constraints file on the horizon container in /openstack/venvs and see if it has the git url in it | 21:59 |
| spatel | looking.. | 21:59 |
| jrosser | and if it does, step back to the repo server and look at the wheel build log to see if it ever built the barbican-ui wheel | 21:59 |
| jrosser | the barbican-ui patch was backported to stable/victoria so if there is a bug with it you maybe hit it now on your point upgrade | 22:01 |
| spatel | what file in - /openstack/venvs/horizon-22.1.2.dev1 | 22:01 |
| jrosser | from memory just look in /openstack/venvs? | 22:02 |
| spatel | nothing side /openstack/venv/ | 22:03 |
| jrosser | hmm sorry i'm not able to vpn now to my stuff | 22:03 |
| spatel | I am not seeing barbican-ui in wheel logs | 22:04 |
| jrosser | oh i do have an AIO though | 22:04 |
| spatel | let me remove wheel logs and re-try | 22:05 |
| jrosser | hold on | 22:06 |
| jrosser | here look this is keystone http://paste.openstack.org/show/804532/ | 22:06 |
| jrosser | so previously we've been over a lot of what happens on the repo server, the generation of all those .txt files | 22:07 |
| spatel | let me check | 22:08 |
| jrosser | this is the other half of that story, about what happens on the target when you want to install the already built wheels from the repo server into a venv | 22:08 |
| spatel | This is what i have http://paste.openstack.org/show/804534/ | 22:09 |
| spatel | may be we can take a look tomorrow, not sure what is going on. | 22:12 |
| jrosser | yeah late here | 22:13 |
| spatel | same here :) | 22:13 |
| spatel | lets talk tomorrow this is lab so nothing urgent | 22:14 |
| spatel | Thank for the help.. i will try to poke tonight and see if i can find something. | 22:14 |
| jrosser | sure, have a good evening | 22:14 |
| spatel | you too.. | 22:14 |
| *** spatel has quit IRC | 22:20 | |
| *** tosky has quit IRC | 22:34 | |
| *** macz_ has quit IRC | 23:16 | |
| *** macz_ has joined #openstack-ansible | 23:20 | |
| *** macz_ has quit IRC | 23:25 | |
| *** spatel_ has joined #openstack-ansible | 23:57 | |
| *** spatel_ is now known as spatel | 23:57 | |
| *** luksky has quit IRC | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!