Thursday, 2021-07-01

jrossernoonedeadpunk: should we merge https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/798881 even though theres no tests05:27
jrosserit fails on master due to OVN - metadata agent is somehow broken05:28
*** chandankumar is now known as chkumar|rover05:31
noonedeadpunkjrosser: first of all that's not needed on master07:02
jrosseryeah, i know07:03
jrosserwas trying to have something in place for when we branch X07:03
noonedeadpunkand I think we should just generally filter all u-c now since they're unreliable anymore07:04
noonedeadpunkfor now I set sha for requirements repo prior to adding neutron there07:05
jrosserah ok07:05
noonedeadpunkprometheanfire: well I actually thought, that u-c are named that way to define a dependencies for the projects, and not include projects themselves. 07:08
noonedeadpunknow they make less sense to me and can't be thought as external dependencies anymore07:09
noonedeadpunkso we're kind of removed lower constraints and messed up upper ones...07:09
noonedeadpunkwhich is weird imo07:10
prometheanfirenoonedeadpunk: it's to lock all librarys used by openstack07:12
prometheanfireunfortunately some openstack projects use other service projects as libraries07:12
prometheanfire(neutron-plugins and horizon plugins I think)07:12
jrosserit seems to me that we ar using u-c now as a sticking plaster for problems in projects07:13
noonedeadpunkI mean if you want to install openstack - you can't use u-c as it would mess up all things07:13
jrosserthe bar used to (imho) be much higer than this for what was acceptable for u-c07:13
prometheanfireis the problem mainly neutron or something else more systemic?07:14
noonedeadpunkI agree, instead of doing that kind of things, lets revive lower constraints then07:14
jrosserthe systemic problem is that we allow top level projects into u-c at all07:14
noonedeadpunkneutron, horizon, tempest, etc07:14
jrosserwhen there already exists neutron-lib for exactly the thing they need07:14
prometheanfirejrosser: I don't think I had much of a choice :|07:14
prometheanfireneutron-lib came after projects used neutron as a lib07:14
noonedeadpunkthe problem is that we also don't have a choice then use u-c, which now tends to become more and more wierd...07:15
prometheanfireUC should be a thing that provides a single co-installable openstack that passes tests07:16
prometheanfireiirc that's the only target it's ever been stated to be07:16
jrosserprometheanfire: it's setting a big precedent, and there was not obvious discussion in the patch adding neutron to u-c, (nor a related bug?)07:16
prometheanfireasperationally I've always wanted it to be more (for packagers, lower-constaints, etc), but always work (for downstream projects too)07:17
prometheanfireneutron has been in UC for stable branches for at least two years iirc07:17
prometheanfireprocess has been, soon after stable release, it gets added to UC of that branch07:17
jrosserwell, the new pip resolver has new opinions about this unfortunatley07:18
prometheanfireit's left out of master as master is dev07:18
noonedeadpunkwell, I mean we have close to no choice now except to maintain own copy of uc that will be filtered from all nasty stuff put there 07:18
prometheanfirenoonedeadpunk: do you have a list (non-exhaustive) of the 'nasty stuff'?07:18
noonedeadpunkwhich I the way I don't want to go07:18
noonedeadpunkall openstack projects07:19
prometheanfireI feel like the pip resolver issue is more pushing the original issue of using service projects as libs07:19
prometheanfirenoonedeadpunk: oslo included?07:19
noonedeadpunknope07:19
noonedeadpunksorry, I meant more like services I guess07:19
noonedeadpunkoslo is really a dependency07:20
prometheanfireok, so far I think that should only be neutron and horizon07:20
noonedeadpunklike neutron-lib07:20
prometheanfireand ya, neutron plugins should be using neutron-lib07:20
jrosserceilometer is there too i think07:20
prometheanfireis that still technically openstack? :P07:20
prometheanfirebut ya, point taken07:20
prometheanfirethis is poking at a wound that openstack has had for 3-4 years at least07:21
prometheanfireI'd say every 2-3 cycles I try and bring it up that plugins need to stop depending on service projects as libs, then get a 'ya, ok, we'll get to it' then it's another few cycles07:22
prometheanfiremaybe I should have picked that rather than lower constraints last time07:22
prometheanfirewell, now pip resolver has come along and kicked over our sandcastle07:23
jrosserperhaps we need an auto generated “clean” version of the u-c file in the same repo07:23
noonedeadpunkreturn these to some another "blacklist" and filter out?07:24
prometheanfireI thought OSA had a way to sed out a list of constraints?07:24
noonedeadpunknew resolver doesn't allow to place constraints like we used to do07:24
prometheanfirethe constraints are needed for gate to function07:24
noonedeadpunkso these things need to be in requirements now07:24
noonedeadpunkwhich conflict with constraints07:25
prometheanfireok, I don't understand07:25
* noonedeadpunk dropped test bed with sample....07:25
prometheanfireremoving neutron and the rest from upper-constraints isn't enough?07:25
noonedeadpunkit is07:25
prometheanfireand OSA doesn't have the ability to do that (I thought it has had that for a couple of years)?07:26
noonedeadpunkbut I mean we're continue adding projects and that filtering is kind of nasty and we're continue adding new services to uc07:26
noonedeadpunklike https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/798960/1/tasks/neutron_install.yml 07:27
noonedeadpunkbut it means we need to start maintaining the list of things that shouldn't be in u-c at the first place07:28
prometheanfirehmm, I thought there was a centralized list or something07:28
noonedeadpunknope not yet07:28
noonedeadpunkas it was limited only to just specific things07:29
noonedeadpunkbut feels like it's more practise now rather then exceptions07:29
prometheanfireok, so, this sucks, but.... requirements as a project exists for openstack gate as the 'client'  packaging and deployment support is a happy accident07:29
prometheanfirethat said, mailing list, maybe more pressure can cause others to stop using neutron as a lib07:30
prometheanfireas far as I know, only neutron ceilometer and horizon are used this way07:30
prometheanfirenothing new has been added in years07:30
prometheanfire(as in at least 2)07:31
noonedeadpunkneutron has been added month ago?07:31
prometheanfireto a NEW stable branch07:31
prometheanfirewhen the next stable branch comes it, it will be added AFTER the branch, not to master before the branch07:31
noonedeadpunkah, well, maybe we didn't saw that because used older resolver...07:31
prometheanfireor it was added a bit later than normal07:31
prometheanfire(maybe a month rather than a week after branching07:32
prometheanfirereally though, I should 'forget' to add it more often, causing pain for those that use service projects as libs is a good thing (change that behavior)07:32
*** sshnaidm is now known as sshnaidm|afk09:24
noonedeadpunkjrosser: let's merge https://review.opendev.org/c/openstack/openstack-ansible/+/798901 ?09:57
jrosseryes - thats with u-c pinned back?09:58
noonedeadpunkyep09:58
jrossercool09:58
noonedeadpunklet's solve u-c thing when we will have CI in place?09:58
noonedeadpunkalso we have so much s**t on master to cover...09:59
*** sshnaidm|afk is now known as sshnaidm10:47
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_mount master: Add integrated linters test  https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/79903611:22
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_service master: Add integrated linters test  https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/79903711:24
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_networkd master: Add integrated linters test  https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/79903811:25
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-apt_package_pinning master: Add integrated linters test  https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/79903911:27
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_server master: Add integrated linters test  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79904111:42
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Start Xena cycle  https://review.opendev.org/c/openstack/openstack-ansible/+/79892011:57
opendevreviewMerged openstack/openstack-ansible master: Create stable/wallaby branch  https://review.opendev.org/c/openstack/openstack-ansible/+/79890112:13
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible/+/79907612:55
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible/+/79907712:55
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible master: Update master for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible/+/79907812:55
opendevreviewMerged openstack/openstack-ansible master: Update master for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible/+/79907813:30
opendevreviewMerged openstack/openstack-ansible stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible/+/79907613:39
opendevreviewMerged openstack/openstack-ansible stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible/+/79907713:39
opendevreviewGeorgina Shippey proposed openstack/openstack-ansible-os_keystone master: Improvements to federation packaging  https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/79891114:22
noonedeadpunkjrosser: do you remeber what was the issue when we had in /openstack/venvs/nova-23.0.0.0b2.dev31/constraints.txt just argument instead of actually constraints?15:02
noonedeadpunkI can recall you was patching it one day15:03
jrosserdo you have an example?15:04
opendevreviewMerged openstack/openstack-ansible-tests stable/wallaby: Fix repo passing CI  https://review.opendev.org/c/openstack/openstack-ansible-tests/+/79866415:04
noonedeadpunkyeah, it's all https://review.opendev.org/c/openstack/openstack-ansible/+/79892015:05
noonedeadpunkhave an aio though15:05
opendevreviewMerged openstack/openstack-ansible-galera_server master: Improve support for tags  https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/79885315:06
noonedeadpunkwith exact same IP as all previous ones :)15:07
noonedeadpunkhttp://paste.openstack.org/show/807108/15:08
noonedeadpunkeventually if I do `curl http://172.29.236.101:8181/constraints/upper_constraints_98091329cf49a3abc4840343401711bdb4f05bd3.txt > /openstack/venvs/nova-23.0.0.0b2.dev31/constraints.txt` it will work15:10
jrosserwell, is this lxc or metal?15:11
noonedeadpunkmetal15:11
jrosserso theres no repo server15:11
jrosserthe constraints file gets populated with the wheels that get built15:12
noonedeadpunkno dedicated one, but `http://172.29.236.101:8181/constraints/upper_constraints_98091329cf49a3abc4840343401711bdb4f05bd3.txt` works as a link15:12
jrosseroh right - so u-c as a URL in constraints.txt doesnt work, but curl the contents and into constraints.txt and it does??15:13
noonedeadpunkyep15:13
noonedeadpunkI think instead of `/openstack/venvs/nova-23.0.0.0b2.dev31/constraints.txt` it should be `http://172.29.236.101:8181/constraints/upper_constraints_98091329cf49a3abc4840343401711bdb4f05bd3.txt`15:14
noonedeadpunkand I recall us patching it...15:14
noonedeadpunkbut can't recall how/where15:14
jrosserwell this is wierd https://github.com/openstack/requirements/blob/stable/wallaby/upper-constraints.txt#L39015:15
jrosservs. ERROR: Could not find a version that satisfies the requirement oslo.config>=8.6.0 (from nova)15:15
jrosseroh15:16
noonedeadpunkum, it's patch switching from W to master15:16
jrosserdoh this is master isnt it15:16
noonedeadpunkit is :(15:16
jrosserwhats the requiements SHA pointing to here https://review.opendev.org/c/openstack/openstack-ansible/+/798920/4/playbooks/defaults/repo_packages/openstack_services.yml15:19
noonedeadpunkdoh15:19
noonedeadpunksorry for taking your time15:20
noonedeadpunkI bet I was checking that15:20
jrossernp :)15:20
*** sshnaidm is now known as sshnaidm|afk15:46
opendevreviewMerged openstack/openstack-ansible-plugins stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/79859615:56
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/79846516:02
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-tests master: Add ansible-role-pki to required_projects  https://review.opendev.org/c/openstack/openstack-ansible-tests/+/79910416:38
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-tests master: Add ansible-role-pki to required-projects  https://review.opendev.org/c/openstack/openstack-ansible-tests/+/79910416:38
noonedeadpunkwell, ansible lint is not going to pass anyway....17:02
noonedeadpunkfor https://review.opendev.org/q/topic:%22create-wallaby%22+(status:open)17:02
noonedeadpunknext plan - merge https://review.opendev.org/q/topic:%22integrated_linters%22+(status:open) and backport :(17:03
noonedeadpunkwworth starting from https://review.opendev.org/c/openstack/openstack-ansible-tests/+/79881417:05
jrosserlinter passes on master for 79910417:08
jrosserwhats up with -w role-name not seeming to be working17:11
noonedeadpunkit's not that17:11
noonedeadpunkit's about ansible-lint changes17:11
noonedeadpunkso, when ansible-lint is run, it symlinks current repo to the `/root/.cache/ansible-lint/603ce4/roles`17:12
noonedeadpunkbut seems with 5.0.0 regexp is failing for us17:12
noonedeadpunkhttp://paste.openstack.org/show/807115/17:12
jrosseroh i didnt spot the internal error17:14
noonedeadpunkfor `ansible-role-qdrouterd` it symlinks to role-qdrouterd17:14
noonedeadpunkwhich is stupid kind of but probably works for somebody 17:16
noonedeadpunkand no idea how to fix that except finishing transfer to integrated linters17:16
noonedeadpunkwell - i have idea - fix ansible-lint regexp, but meh17:18
noonedeadpunkoh, wait17:29
noonedeadpunkwe can workaround by adding role_name to meta17:30
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_server stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79860917:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-tests stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-tests/+/79861317:33
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_server stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79861017:34
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_service stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/79845317:35
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-apt_package_pinning stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/79846017:36
jrosserso we merge and backport https://review.opendev.org/c/openstack/openstack-ansible-tests/+/799104 ?17:36
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_networkd stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/79845017:37
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/79846317:37
noonedeadpunkum, yes, but I don't think it's that required once pki is in tests/a-r-r17:39
noonedeadpunkfor these roles17:39
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_server stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79861017:40
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_server stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79861017:40
noonedeadpunkdamn, why I haven't spotted that before branching integrated role...17:41
noonedeadpunkwhile there were no tests to run17:41
noonedeadpunkdamn.... bump tool got broken 2 weeks ago :(17:45
ThiagoCMCHey guys! How to deal with RabbitMQ consuming all CPU cores?17:48
ThiagoCMCIs there any alternative? I mean, is it possible to deploy OSA with something else entirely? This RabbitMQ thing sucks.17:49
noonedeadpunkthe question should be is it possible to deploy openstack with smth other the rabbitmq17:49
noonedeadpunkanswer is - partially.17:49
noonedeadpunksmth can be covered with qrouterd17:49
noonedeadpunkand, we have a role for it - not sure it has been used by anybody  though17:50
noonedeadpunkand we not payed any attention to it for years17:50
noonedeadpunkbut you can try:)17:50
noonedeadpunkah, seems you can use kafka as well nowadays17:52
noonedeadpunkhttps://docs.openstack.org/oslo.messaging/latest/admin/drivers.html17:52
noonedeadpunkso instead of rabbit you can deploy technically qrouterd and kafka17:53
noonedeadpunkdoh, time to move aways from travis :(18:12
noonedeadpunkand learn more about github actions18:15
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-apt_package_pinning stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/79846018:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-apt_package_pinning stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/79846118:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_server stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79860918:29
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_server stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79861018:30
opendevreviewMerged openstack/openstack-ansible-apt_package_pinning stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/79846019:03
opendevreviewMerged openstack/openstack-ansible-tests stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-tests/+/79861319:13
opendevreviewMerged openstack/openstack-ansible-apt_package_pinning stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/79846119:16
ThiagoCMCIt would be really nice to be able to at least try something else! I'm really tired of dealing with RabbitMQ issues.19:40
ThiagoCMCI tried that hybrid approach few months ago, it failed.19:41
noonedeadpunkThiagoCMC: there was some talk recently about scaling rabbitmq...20:44
noonedeadpunkhttp://lists.openstack.org/pipermail/openstack-discuss/2021-March/021121.html20:46
noonedeadpunkI bet there was follow up email with recording of it20:47
noonedeadpunkbut eventually it was all coming down to disabling ha queues and adding dedicated hosts for rabbit20:48
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/79846320:49
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/79846520:49
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_networkd stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/79845020:50
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_networkd stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/79845120:50
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_service stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/79845320:51
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_service stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/79845420:51
noonedeadpunkThiagoCMC: also - maybe you have some queues that are not read? Ie notification ones?20:53
noonedeadpunkas it may bring huge troubles, when some queues have hundreds of thousands unread messages20:53
ThiagoCMCnoonedeadpunk, that might be it! I just don't know exactly how to clean it up.21:12
opendevreviewMerged openstack/openstack-ansible-rsyslog_server stable/wallaby: Update .gitreview for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79860922:08
opendevreviewMerged openstack/openstack-ansible-rsyslog_server stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/79861022:08

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!