| openstackgerrit | Merged openstack/openstack-ansible-tests stable/queens: Add ceph-ansible library and callback path https://review.openstack.org/587198 | 00:06 |
|---|---|---|
| *** chason has joined #openstack-ansible | 01:22 | |
| *** mattt1 has joined #openstack-ansible | 01:28 | |
| *** kaiokmo has quit IRC | 01:33 | |
| *** spatel has joined #openstack-ansible | 01:42 | |
| *** kaiokmo has joined #openstack-ansible | 02:19 | |
| *** spatel has quit IRC | 03:36 | |
| *** spatel has joined #openstack-ansible | 03:36 | |
| *** spatel has quit IRC | 03:37 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Unpin ceph-ansible and add ceph-ansible library path https://review.openstack.org/587200 | 03:40 |
| *** dxiri has joined #openstack-ansible | 03:42 | |
| *** udesale has joined #openstack-ansible | 03:52 | |
| *** dxiri has quit IRC | 03:54 | |
| *** dxiri has joined #openstack-ansible | 03:55 | |
| *** d34dh0r53 has quit IRC | 04:00 | |
| *** cloudnull has quit IRC | 04:00 | |
| *** eglute_s has quit IRC | 04:00 | |
| *** eglute has joined #openstack-ansible | 04:04 | |
| *** d34dh0r53 has joined #openstack-ansible | 04:04 | |
| *** d34dh0r53 has quit IRC | 04:05 | |
| *** eglute has quit IRC | 04:05 | |
| *** d34dh0r53 has joined #openstack-ansible | 04:06 | |
| *** eglute has joined #openstack-ansible | 04:06 | |
| *** cloudnull has joined #openstack-ansible | 04:11 | |
| *** dxiri has quit IRC | 04:15 | |
| *** dxiri has joined #openstack-ansible | 04:20 | |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts master: Add route when required on the local table https://review.openstack.org/588406 | 04:53 |
| *** shyambiradar has joined #openstack-ansible | 05:03 | |
| *** viks_ has joined #openstack-ansible | 05:10 | |
| *** galaxyblr has joined #openstack-ansible | 05:11 | |
| viks_ | +cnrtf | 05:19 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible master: zuul: Add aio_lxc jobs for distribution installations https://review.openstack.org/584765 | 05:28 |
| *** hwoarang has joined #openstack-ansible | 05:28 | |
| *** shyambiradar has quit IRC | 05:33 | |
| *** udesale has quit IRC | 05:34 | |
| *** shyambiradar has joined #openstack-ansible | 05:47 | |
| *** dxiri has quit IRC | 05:49 | |
| *** Pradiprwt has joined #openstack-ansible | 06:03 | |
| *** nicolasbock has joined #openstack-ansible | 06:05 | |
| Pradiprwt | Hello Everyone, Good Morning | 06:05 |
| *** pcaruana has joined #openstack-ansible | 06:07 | |
| *** udesale has joined #openstack-ansible | 06:11 | |
| *** luksky has joined #openstack-ansible | 06:42 | |
| *** ivve has quit IRC | 06:46 | |
| *** ivve has joined #openstack-ansible | 06:46 | |
| Pradiprwt | Guys, I facing this issue >> http://paste.openstack.org/show/727382/ Is it known issue?? | 06:54 |
| *** jhesketh_ has joined #openstack-ansible | 07:10 | |
| *** cshen has joined #openstack-ansible | 07:11 | |
| *** jhesketh has quit IRC | 07:11 | |
| *** DanyC has joined #openstack-ansible | 07:12 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_aodh master: Trivial: Fix the pep8 warning https://review.openstack.org/589029 | 07:13 |
| *** rmart04 has joined #openstack-ansible | 07:14 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_barbican master: Trivial: Fix the pep8 warning https://review.openstack.org/589031 | 07:16 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_blazar master: Trivial: Fix the pep8 warning https://review.openstack.org/589032 | 07:17 |
| *** DanyC has quit IRC | 07:17 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_ceilometer master: Trivial: Fix the pep8 warning https://review.openstack.org/589033 | 07:19 |
| *** Bhujay has joined #openstack-ansible | 07:21 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_cinder master: Trivial: Fix the pep8 warning https://review.openstack.org/589035 | 07:21 |
| *** gillesMo has joined #openstack-ansible | 07:22 | |
| *** spatel has joined #openstack-ansible | 07:22 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_cloudkitty master: Trivial: Fix the pep8 warning https://review.openstack.org/589036 | 07:23 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_glance master: Trivial: Fix the pep8 warning https://review.openstack.org/589038 | 07:29 |
| *** DanyC has joined #openstack-ansible | 07:29 | |
| *** Bhujay has quit IRC | 07:30 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_gnocchi master: Trivial: Fix the pep8 warning https://review.openstack.org/589040 | 07:32 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_heat master: Trivial: Fix the pep8 warning https://review.openstack.org/589041 | 07:33 |
| *** spatel has quit IRC | 07:34 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_ironic master: Trivial: Fix the pep8 warning https://review.openstack.org/589044 | 07:34 |
| *** DanyC has quit IRC | 07:34 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_magnum master: Trivial: Fix the pep8 warning https://review.openstack.org/589045 | 07:36 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_masakari master: Trivial: Fix the pep8 warning https://review.openstack.org/589046 | 07:37 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_barbican master: [TEST] bionic https://review.openstack.org/586655 | 07:38 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: [TEST] bionic https://review.openstack.org/586657 | 07:38 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: [TEST] bionic https://review.openstack.org/586656 | 07:39 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_molteniron master: Trivial: Fix the pep8 warning https://review.openstack.org/589048 | 07:39 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_designate master: [TEST] bionic https://review.openstack.org/586658 | 07:39 |
| *** Bhujay has joined #openstack-ansible | 07:39 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance master: [TEST] bionic https://review.openstack.org/586659 | 07:39 |
| *** tosky has joined #openstack-ansible | 07:39 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_gnocchi master: [TEST] bionic https://review.openstack.org/586660 | 07:39 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_monasca-agent master: Trivial: Fix the pep8 warning https://review.openstack.org/589049 | 07:40 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_heat master: [TEST] bionic https://review.openstack.org/586661 | 07:40 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_horizon master: [TEST] bionic https://review.openstack.org/586662 | 07:40 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: [TEST] bionic https://review.openstack.org/586663 | 07:41 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_molteniron master: [TEST] bionic https://review.openstack.org/586665 | 07:41 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: [TEST] bionic https://review.openstack.org/586666 | 07:42 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_monasca-ui master: Trivial: Fix the pep8 warning https://review.openstack.org/589050 | 07:42 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: [TEST] bionic https://review.openstack.org/586667 | 07:42 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: [TEST] bionic https://review.openstack.org/586668 | 07:43 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_rally master: [TEST] bionic https://review.openstack.org/586669 | 07:43 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_sahara master: [TEST] bionic https://review.openstack.org/586670 | 07:43 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_swift master: [TEST] bionic https://review.openstack.org/586671 | 07:44 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tacker master: [TEST] bionic https://review.openstack.org/586672 | 07:44 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tempest master: [TEST] bionic https://review.openstack.org/586673 | 07:44 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_neutron master: Trivial: Fix the pep8 warning https://review.openstack.org/589051 | 07:45 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_almanach master: Switch to using project-templates https://review.openstack.org/587531 | 07:45 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_congress master: Switch to using project-templates https://review.openstack.org/587532 | 07:46 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_monasca master: Switch to using project-templates https://review.openstack.org/587533 | 07:46 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_watcher master: Switch to using project-templates https://review.openstack.org/587534 | 07:46 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_zaqar master: Switch to using project-templates https://review.openstack.org/587535 | 07:46 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_nova master: Trivial: Fix the pep8 warning https://review.openstack.org/589053 | 07:47 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_zun master: Switch to using project-templates https://review.openstack.org/587536 | 07:47 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_searchlight master: Switch to using project-templates https://review.openstack.org/587541 | 07:47 |
| *** DanyC has joined #openstack-ansible | 07:49 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_octavia master: Trivial: Fix the pep8 warning https://review.openstack.org/589054 | 07:49 |
| evrardjp | morning | 07:49 |
| jrosser | morning | 07:49 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_panko master: Trivial: Fix the pep8 warning https://review.openstack.org/589055 | 07:51 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_rally master: Trivial: Fix the pep8 warning https://review.openstack.org/589056 | 07:52 |
| *** rpittau has joined #openstack-ansible | 07:53 | |
| *** DanyC has quit IRC | 07:54 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_sahara master: Trivial: Fix the pep8 warning https://review.openstack.org/589057 | 07:54 |
| *** Bhujay has quit IRC | 07:54 | |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_searchlight master: Trvial: Fix the pep8 warning https://review.openstack.org/589058 | 07:55 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_swift master: Trivial: Fix the pep8 warning https://review.openstack.org/589059 | 07:56 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_trove master: Trivial: Fix the pep8 warning https://review.openstack.org/589060 | 07:57 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_watcher master: Trivial: Fix the pep8 warning https://review.openstack.org/589061 | 07:58 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_zaqar master: Trivial: Fix the pep8 warning https://review.openstack.org/589062 | 07:59 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_zun master: Trivial: Fix the pep8 warning https://review.openstack.org/589063 | 08:00 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Do not install linux-image-extra by default https://review.openstack.org/589064 | 08:00 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Do not install linux-image-extra by default https://review.openstack.org/589065 | 08:00 |
| *** DanyC has joined #openstack-ansible | 08:10 | |
| *** DanyC has quit IRC | 08:14 | |
| *** DanyC has joined #openstack-ansible | 08:18 | |
| evrardjp | so | 08:19 |
| evrardjp | this week is the RC week | 08:19 |
| evrardjp | I will try to work with the releases team to branch the roles as a separate thing of the integrated repo, but coordinate the change. | 08:20 |
| evrardjp | if it's possible, from now on, roles would only be branched. | 08:21 |
| evrardjp | if it's not for this cycle, it will be next one | 08:21 |
| evrardjp | should I wait for doing so, because there will be a lot of backports? | 08:21 |
| evrardjp | odyssey4me: ^ | 08:22 |
| evrardjp | cores ^ | 08:22 |
| *** ppetit has joined #openstack-ansible | 08:23 | |
| *** shyambiradar has quit IRC | 08:24 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_almanach master: Use generic vars file for ubuntu https://review.openstack.org/589074 | 08:27 |
| gillesMo | Hello, does someone use Queens and ceilometer and found a workaround this bug ? https://bugs.launchpad.net/openstack-ansible/+bug/1782357 | 08:27 |
| openstack | Launchpad bug 1782357 in openstack-ansible "Undefined ceilometer variables for additional services" [Medium,Confirmed] | 08:27 |
| *** ppetit has quit IRC | 08:27 | |
| *** ppetit has joined #openstack-ansible | 08:27 | |
| Pradiprwt | Hi evrardjp, I facing one issue while installing openstack services.. Here is verbose enabled log http://paste.openstack.org/show/727382/ , Is this known issue? First time i have seen this issue.. | 08:28 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [TEST] bionic https://review.openstack.org/586607 | 08:28 |
| *** ppetit has joined #openstack-ansible | 08:29 | |
| gillesMo | If I should not use ceilometer anymore, what is the alternative ? I can't find a good explanation of the futur/replacement of Celiometer, using Gnocchi, Pakno... | 08:29 |
| gillesMo | If I should not use ceilometer anymore, what is the alternative ? I can't find a good explanation of the futur/replacement of Ceilometer, using Gnocchi, Panko... | 08:29 |
| evrardjp | Pradiprwt: unable to connect to database, check login_user and login_password are correct or /root/.my.cnf has the credentials. Exception message: (2013, \'Lost connection to MySQL server | 08:32 |
| evrardjp | Pradiprwt: which branch? Are you sure your networking is fine? | 08:32 |
| Pradiprwt | pike | 08:32 |
| evrardjp | no change in Pike, it must be your environment | 08:32 |
| evrardjp | gillesMo: why not using ceilometer? | 08:33 |
| evrardjp | gillesMo: it depends on your needs to. | 08:33 |
| gillesMo | evrardjp: I though having read that it will be replaced, but perhaps I misunderstood... | 08:33 |
| evrardjp | ceilometer is not deprecated. | 08:33 |
| evrardjp | not that I am aware of | 08:34 |
| evrardjp | where did you read this? | 08:34 |
| Pradiprwt | evrardjp: networking is fine, i have verified | 08:34 |
| evrardjp | gillesMo: maybe using mongo with ceilometer should be dropped | 08:34 |
| evrardjp | gillesMo: but that's my personal opinion | 08:34 |
| Pradiprwt | I haverified the mysql, it is working from galara container | 08:34 |
| evrardjp | Pradiprwt: check your haproxy state. | 08:35 |
| gillesMo | evrardjp: perhpas it's just the fact that it is splited in pieces with gnocchi, aodh, panko... | 08:36 |
| Pradiprwt | evrardjp: I got some connection refused statement in haproxy log >> http://paste.openstack.org/show/727384/ | 08:36 |
| gillesMo | Well, I miss read, it's in fact just the storage part of ceilometer that is deprecated, in favor of gnocchi | 08:38 |
| *** shyambiradar has joined #openstack-ansible | 08:38 | |
| odyssey4me | evrardjp: the lessons from previous release times is that we should branch when the upstream projects branch | 08:39 |
| odyssey4me | man, got a ton of spam over the weekend - the new strategy they have is to open private messages :/ | 08:39 |
| *** Bhujay has joined #openstack-ansible | 08:45 | |
| Pradiprwt | evrardjp: haproxy is in running state | 08:48 |
| odyssey4me | jrosser: I expect you might want to have this merged to queens? https://review.openstack.org/587871 | 08:48 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Use the TESTING_BRANCH env var for constraints https://review.openstack.org/579371 | 08:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Use the TESTING_BRANCH env var for constraints https://review.openstack.org/579371 | 08:50 |
| jrosser | odyssey4me: yes i think so | 08:51 |
| odyssey4me | evrardjp: ^ so, to prepare for the branching - do we want to implement https://review.openstack.org/579371 ? | 08:51 |
| odyssey4me | evrardjp: if we do, then I need to propose a patch to all repositories for those tox changes, otherwise merging that patch will break all role tests | 08:53 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Simplify inserting apt keys into LXC image https://review.openstack.org/588962 | 08:54 |
| Pradiprwt | 3306 | 08:59 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Make run_tests_common idempotent https://review.openstack.org/579372 | 09:00 |
| *** olivierbourdon38 has joined #openstack-ansible | 09:00 | |
| Pradiprwt | The problem seems to be with HAproxy, can any one please help me | 09:00 |
| Pradiprwt | with same configuration local setup is working, problem with client setup | 09:01 |
| evrardjp | odyssey4me: yes so I have a tip for that | 09:03 |
| evrardjp | do you have weechat? | 09:03 |
| evrardjp | for that there is a plugin that auto closes messages | 09:03 |
| evrardjp | so I just had to roll on those messages, to see if this was spam | 09:03 |
| evrardjp | (100% was, sadly) | 09:03 |
| odyssey4me | evrardjp: no, but yeah - if something like that persists I'll investigate something to resolve it | 09:03 |
| evrardjp | and it autoclosed | 09:03 |
| evrardjp | mine was so Ctrl+N N N N N done. | 09:04 |
| evrardjp | (multiply the Ns) | 09:04 |
| odyssey4me | I'd prefer to figure out something that actually works on the znc, so it doesn't even hit the client | 09:04 |
| evrardjp | yeah but it's to confirm what's legit or not that's hard | 09:04 |
| evrardjp | tbh I removed myself from unregistered now | 09:04 |
| evrardjp | I am done with this channel | 09:05 |
| evrardjp | I helped on the first day, but then... why bother. | 09:05 |
| odyssey4me | I don't understand what you just said. | 09:05 |
| evrardjp | odyssey4me: your spam, was it because you were in the unregistered channel? | 09:05 |
| odyssey4me | no | 09:06 |
| evrardjp | oh great. | 09:06 |
| evrardjp | I thought it was because I was in the unregistered channel. | 09:06 |
| *** Taseer has quit IRC | 09:06 | |
| evrardjp | great. | 09:06 |
| evrardjp | anyway | 09:06 |
| odyssey4me | No. It might be from one of the other channels I'm in - not sure. | 09:06 |
| evrardjp | Pradiprwt: you have to investigate why haproxy returns dead, it depends on your case. | 09:06 |
| evrardjp | odyssey4me: or maybe they are parsing logs | 09:07 |
| evrardjp | curious | 09:07 |
| evrardjp | annoying too | 09:07 |
| hwoarang | hello | 09:07 |
| evrardjp | gillesMo: yeah, mongo dropped for gnocchi | 09:07 |
| evrardjp | hwoarang: hello sir! | 09:07 |
| evrardjp | how were you holidays? | 09:07 |
| odyssey4me | o/ hwoarang - welcome back, hope you had a good holiday! | 09:07 |
| hwoarang | yeah it was a much needed break | 09:07 |
| * hwoarang has a gazillion emails to read | 09:08 | |
| evrardjp | hwoarang: world broke during your time, so don't bother trying to catch up -- just continue what you were busy : ) | 09:08 |
| hwoarang | lol | 09:08 |
| evrardjp | hahaha | 09:08 |
| Pradiprwt | evrardjp : I am trying but not able to find the exact problem, can you please have a look to my configuration file >> openstack_user_config.yml http://paste.openstack.org/show/727389/ || interfaces http://paste.openstack.org/show/727388/ | 09:10 |
| evrardjp | odyssey4me: about branching, yeah it's closing in. We don't have the same requirement of time as others though, but I expect things will be done this week thursday. | 09:10 |
| odyssey4me | evrardjp: yes, we don't *have* to, but we *want* to | 09:10 |
| evrardjp | what I mean by "not the same as others": 1) We don't have sync anymore for requirements IIRC 2) I am not sure why we'd need to wait | 09:11 |
| evrardjp | Pradiprwt: I don't have the time to check on that I am sorry. Restart your haproxy, check if it's better. Check your galera middleware too. | 09:11 |
| Pradiprwt | can you please give me some troubleshootong refrence for the same | 09:12 |
| evrardjp | odyssey4me: why would we WANT to wait for others? Things will break because some teams will not branch on time anyway. | 09:12 |
| odyssey4me | evrardjp: ok, it appears some wires are getting crossed here | 09:13 |
| odyssey4me | you mentioned requirements syncing, which reminded me of that as one reason why we made our branches asap after upstream - I guess that may not be a thing any more | 09:14 |
| evrardjp | Pradiprwt: maybe worth checking there: https://docs.openstack.org/openstack-ansible/latest/admin/maintenance-tasks.html | 09:14 |
| odyssey4me | our rules are we have a 2 week trailing period deadline for releasing, so we have the option of waiting - but I'm saying that we have previously learned that it's better not to | 09:14 |
| evrardjp | odyssey4me: yeah. I think world as changed, and we can therefore change things too. | 09:14 |
| odyssey4me | I guess you're now asking for a refresh on that idea. | 09:15 |
| evrardjp | odyssey4me: I remember for RC it was important to do it earliest. | 09:15 |
| evrardjp | I am now querying: Is there a reason to wait? Do you have pending patches? | 09:15 |
| evrardjp | I don't think we should wait. I'd rather do an RC as soon as world as stabilized | 09:16 |
| evrardjp | as our m3 was not great. : ) | 09:16 |
| odyssey4me | Ah ok, well then - I've suggested that https://review.openstack.org/579371 might make it go smoother, because we won't have to rely on patches to point at the right branches. But for that to merge, we need to propose tox changes to every repo. | 09:16 |
| evrardjp | ok that sounds like a nice plan, let me review this. | 09:19 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Remove octavia_requires_pip_packages https://review.openstack.org/588177 | 09:20 |
| *** olivierbourdon38 has quit IRC | 09:23 | |
| Pradiprwt | Thanks evrardjp, I tried the cluster health command for galara, it is fine... My setup is for 3 node 1 controller and 2 compute. I am trying to debug it.. | 09:25 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Remove octavia_requires_pip_packages https://review.openstack.org/588177 | 09:26 |
| evrardjp | odyssey4me: 2 comments on https://review.openstack.org/#/c/579371/5 could you give me your opininon on the second, so that I work on a patch there? | 09:28 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Use the TESTING_BRANCH env var for constraints https://review.openstack.org/579371 | 09:28 |
| *** olivierbourdon38 has joined #openstack-ansible | 09:29 | |
| odyssey4me | evrardjp: yeah, that probably makes sense to do | 09:31 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Make run_tests_common idempotent https://review.openstack.org/579372 | 09:33 |
| odyssey4me | evrardjp: ok, I need to switch focus to my day job - will switch back to OSA this evening | 09:34 |
| odyssey4me | evrardjp: oh, if you have some time, I'd really appreciate another set of eyes looking at the failure in https://review.openstack.org/582296 - I can't see the problem | 09:38 |
| odyssey4me | gret patch jrosser https://review.openstack.org/588962 | 09:40 |
| odyssey4me | *great | 09:40 |
| jrosser | odyssey4me: the first rule of gpg club is to not use gpg if at all possible :) | 09:42 |
| evrardjp | jrosser: https://www.youtube.com/watch?v=kmL5EkvFYPo | 09:48 |
| openstackgerrit | Merged openstack/openstack-ansible master: Change command for shell https://review.openstack.org/587638 | 09:49 |
| jrosser | evrardjp: :) | 09:54 |
| Pradiprwt | I have verified everything but still I am geeting the issue >>> fatal: [infra1_keystone_container-68685a88 -> 36.101.13.118]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} | 09:55 |
| jrosser | evrardjp: could you give this one a look? https://review.openstack.org/#/c/588962/ | 09:56 |
| odyssey4me | jrosser evrardjp top tip, set umode +R for your IRC client to only accept private messages from registered users | 09:56 |
| odyssey4me | ie type /umode +R | 09:56 |
| odyssey4me | see https://forums.linuxmint.com/viewtopic.php?t=215223 | 09:57 |
| odyssey4me | umode +c did not work for me | 09:58 |
| odyssey4me | evrardjp: regarding your comment/review in https://review.openstack.org/#/c/582296/ - yes, I saw that error, but I can't understand why it's hitting that error because it seems to me the tasks are creating that domain | 09:59 |
| odyssey4me | see http://logs.openstack.org/96/582296/16/check/openstack-ansible-functional-ubuntu-xenial/c3557cf/logs/ara-report/result/a75df51d-d8a6-4fe0-8e13-8a0a5b64816f/ | 10:01 |
| evrardjp | odyssey4me: I think it's the '.' | 10:02 |
| evrardjp | I will investigate later | 10:02 |
| evrardjp | I have to go | 10:02 |
| evrardjp | odyssey4me: I planned to use domain_facts to see | 10:03 |
| evrardjp | I will do that later | 10:03 |
| odyssey4me | ok, thanks - chat later then | 10:03 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582296 | 10:06 |
| odyssey4me | evrardjp: I spotted a small difference there which may resolve it. | 10:08 |
| evrardjp[m] | cook | 10:08 |
| evrardjp[m] | cool* | 10:08 |
| evrardjp | odyssey4me: haha good catch | 10:13 |
| Adri2000 | here is a bug report https://bugs.launchpad.net/openstack-ansible/+bug/1785592 for (I think) a dynamic inventory bug that caused my pike->queens upgrade from last week to not install neutron agents on bare metal as expected (I brought up that issue here last week :)) | 10:14 |
| openstack | Launchpad bug 1785592 in openstack-ansible "dynamic inventory doesn't handle is_metal: false->true change" [Undecided,New] | 10:14 |
| Pradiprwt | Aug 6 04:04:09 localhost haproxy[20191]: Server keystone_service-back/infra1_keystone_container-68685a88 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue. | 10:23 |
| Pradiprwt | I am getting this error for all containers in haproxy log, is it known issue. I am not able to understand what I am missing | 10:24 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_almanach master: Switch to using project-templates https://review.openstack.org/587531 | 10:25 |
| *** shyambiradar has quit IRC | 10:28 | |
| *** luksky has quit IRC | 10:28 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_watcher master: Switch to using project-templates https://review.openstack.org/587534 | 10:28 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_zaqar master: Switch to using project-templates https://review.openstack.org/587535 | 10:29 |
| *** noonedeadpunk has joined #openstack-ansible | 10:29 | |
| noonedeadpunk | Hi guys | 10:30 |
| *** hughsaunders has quit IRC | 10:32 | |
| *** cshen has quit IRC | 10:33 | |
| *** udesale has quit IRC | 10:34 | |
| *** spatel has joined #openstack-ansible | 10:35 | |
| *** hughsaunders has joined #openstack-ansible | 10:35 | |
| evrardjp | Adri2000: oh great. | 10:36 |
| evrardjp | Pradiprwt: that's a network issue. L4. tcp connection failure. | 10:36 |
| evrardjp | check your networking. | 10:36 |
| evrardjp | noonedeadpunk: hey | 10:37 |
| Pradiprwt | evrardjp: I am really struglling with thiss issue from friday, not able to find what I am missing... Another setup is working fine :( | 10:38 |
| *** spatel has quit IRC | 10:39 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 10:41 |
| noonedeadpunk | great, I'm in. I wasn't registered in freenode, and while I was in vacation registration become required... | 10:42 |
| jrosser | Adri2000: is this patch applied? https://review.openstack.org/#/c/528357/ | 10:43 |
| *** shyambiradar has joined #openstack-ansible | 10:46 | |
| *** dave-mccowan has joined #openstack-ansible | 10:47 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Remove octavia_requires_pip_packages https://review.openstack.org/588177 | 10:50 |
| Adri2000 | jrosser: yes it is | 10:55 |
| jrosser | Adri2000: your trouble feels very much related to what that patch needed to fix for other parts of the inventory | 10:56 |
| jrosser | that change might point you toward the relevant part of the code | 10:57 |
| *** vnogin has joined #openstack-ansible | 10:57 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_trove master: [TEST] bionic https://review.openstack.org/586674 | 10:57 |
| *** luksky has joined #openstack-ansible | 11:02 | |
| *** ansmith has quit IRC | 11:03 | |
| *** dave-mccowan has quit IRC | 11:03 | |
| *** shyambiradar has quit IRC | 11:05 | |
| *** cshen has joined #openstack-ansible | 11:06 | |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_masakari master: Basic implemintation of masakari-monitors https://review.openstack.org/584629 | 11:07 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-galera_client master: Add retries for apt https transport installation https://review.openstack.org/589118 | 11:10 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-galera_client master: List ubuntu releases in release order https://review.openstack.org/589119 | 11:11 |
| *** shyambiradar has joined #openstack-ansible | 11:13 | |
| *** shyambiradar has quit IRC | 11:30 | |
| odyssey4me | noonedeadpunk: not sure if you meant to revert all those changes, but before modifying the patch again you should have updated your fork | 11:31 |
| noonedeadpunk | odyssey4me: Yep, I'm already returning them back... sorry | 11:33 |
| odyssey4me | noonedeadpunk: heh, no worries - not sure if you're aware of this, but you can download patch set 15 using git review -d 584629,15 | 11:34 |
| noonedeadpunk | odyssey4me: yep, I'm aware, just a bit confused with squashing commits - I was going just to resolve conflicts with already merged patches | 11:37 |
| odyssey4me | noonedeadpunk: PS15 had no merge conflicts - I had just resolved them all | 11:38 |
| odyssey4me | unless you've done further work on top? | 11:38 |
| evrardjp | Pradiprwt: I am not able to tell you more than that with what you said | 11:39 |
| *** mmalchuk has joined #openstack-ansible | 11:39 | |
| evrardjp | Pradiprwt: as I said, check what's in haproxy for the healthcheck of galera for your version. Do exactly as haproxy, see what fails. | 11:39 |
| evrardjp | fix it, and your cluster will be back alive | 11:40 |
| evrardjp | odyssey4me: how nice of you to fix the merge conflicts for others : ) | 11:40 |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 11:41 |
| noonedeadpunk | odyssey4me: oh... than I just created some mess.... reverted everything back. thatnks for this) | 11:43 |
| *** suggestable has joined #openstack-ansible | 11:43 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_trove master: [TEST] bionic https://review.openstack.org/586674 | 11:46 |
| odyssey4me | noonedeadpunk: :) apologies, I probably should have left you to it - the patch still isn't in a working state, and I wanted the rebase to get a fresh result to try and see why | 11:47 |
| evrardjp | odyssey4me: noonedeadpunk I see no problem when things are done with best intentions :) | 11:52 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_swift master: vars: Add missing python-keystonemiddleware dependency https://review.openstack.org/589128 | 11:53 |
| hwoarang | could i get a quick review on ^^^ please? | 11:53 |
| evrardjp | jrosser: odyssey4me ^ | 11:55 |
| *** pdeters has joined #openstack-ansible | 11:55 | |
| jrosser | i'm just wondering why those seem to have passed my tests without needing that? | 11:55 |
| hwoarang | it's possible that something else could have pulled them in | 11:56 |
| jrosser | this is mostly green https://review.openstack.org/#/c/586671/ | 11:57 |
| hwoarang | but opensuse aio fails without that | 11:57 |
| jrosser | ok :) | 11:57 |
| jrosser | done | 11:58 |
| hwoarang | thank you both | 11:58 |
| * hwoarang is ready to hit the magic 'recheck' button | 11:58 | |
| jrosser | can i swap you a look at this? https://review.openstack.org/#/c/588962/ | 11:58 |
| odyssey4me | folks, I could do with another set of eyes on https://review.openstack.org/587871 please - it helps offline installs | 12:00 |
| noonedeadpunk | evrardjp: fully agree with you. odyssey4me: it's okey. the question is, why lxc containers stopped to be created correctly.... | 12:00 |
| hwoarang | both done | 12:04 |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 12:04 |
| evrardjp | okay folks I will focus on messing up things for RC with releases team | 12:08 |
| kaiokmo | morning | 12:08 |
| noonedeadpunk | Can someone please take a look at the following changes: https://review.openstack.org/#/c/586318/ https://review.openstack.org/#/c/583943/ ? | 12:10 |
| Pradiprwt | evrardjp: Here I did telnet and verified galera clustur status, http://paste.openstack.org/show/727409/ | 12:11 |
| Pradiprwt | evrardjp: How to check haproxy for the healthcheck of galera ? | 12:12 |
| evrardjp | check the haproxy.cfg | 12:12 |
| evrardjp | /etc/haproxy/ | 12:12 |
| jamesdenton | evrardjp if you get some time, https://review.openstack.org/#/q/owner:%22James+Denton%22+status:open+branch:master+(bug/1782625) | 12:13 |
| evrardjp | will do, but I don't have time, I will MAKE some time for you :) | 12:13 |
| jamesdenton | What a friend ;) | 12:14 |
| evrardjp | jamesdenton: hahaha | 12:14 |
| *** shyambiradar has joined #openstack-ansible | 12:15 | |
| evrardjp | cloudnull: could you have a careful look at https://review.openstack.org/#/c/584447/1 ? I checked it very quickly and it seemed okay to me, but the more eyes, the better. | 12:17 |
| Pradiprwt | evrardjp: Where I can see healthchek logs ? | 12:18 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Simplify inserting apt keys into LXC image https://review.openstack.org/588962 | 12:19 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible master: zuul: Add aio_lxc jobs for distribution installations https://review.openstack.org/584765 | 12:20 |
| jrosser | i have a handful ready to go here if anyone has a moment https://review.openstack.org/#/q/topic:bionic-check+(status:open+OR+status:merged) | 12:24 |
| jrosser | evrardjp: when shall we make bionic role tests voting? ~75% are passing and it would be good to prevent regressions in those | 12:27 |
| Pradiprwt | cloudnull: Hi | 12:35 |
| noonedeadpunk | I've faced with interesting bug, which affects nova and ceph (and, ceph_client role). https://bugs.launchpad.net/nova/+bug/1777123 So the point is, that python-ceph package installs obsolete rbd library for python, which is linked into nova venv. Probably we should consider building rbd from sources rather than installing by apt? | 12:39 |
| openstack | Launchpad bug 1777123 in OpenStack Compute (nova) "Nova 17.04 fails to create ephemeral storage with rbd driver: TypeError: name must be a string" [Low,Confirmed] - Assigned to melanie witt (melwitt) | 12:39 |
| *** spatel has joined #openstack-ansible | 12:40 | |
| Pradiprwt | Hi odyssey4me, can you please help me to make my galara cluster up.. | 12:40 |
| odyssey4me | noonedeadpunk: those libs have c bindings, so trying to compile them from source is messy and inconsistent from one platform to the next - that's why we use the distro package instead | 12:41 |
| odyssey4me | Pradiprwt: no, sorry, I cannot | 12:42 |
| hwoarang | jrosser: could you upgrade your vote on https://review.openstack.org/#/c/583919/ please? :) | 12:43 |
| hwoarang | or odyssey4me for that matter | 12:43 |
| jrosser | done | 12:43 |
| hwoarang | thank you! | 12:44 |
| openstackgerrit | Merged openstack/openstack-ansible master: Add more systematic healthchecks https://review.openstack.org/587408 | 12:44 |
| openstackgerrit | Merged openstack/openstack-ansible master: Use ansible module instead of command https://review.openstack.org/587589 | 12:44 |
| ivve | hi guys, any idea when this error will be merged to pike/stable, its known.. AttributeError: 'Guest' object has no attribute 'migrate_configure_max_speed' during live migration | 12:45 |
| ivve | i just got the latest a few hours ago and it was not in there (did a minor upgrade) | 12:46 |
| ivve | i was under the impression it was :) | 12:46 |
| odyssey4me | ivve: are you asking when a patch will be merged? we try not to merge errors ;) | 12:47 |
| ivve | its nova <-> ceph guest.py, the attribute is simply missing. is there in queens i believe. closed down all tabs for the bug | 12:47 |
| ivve | ah yes :D | 12:47 |
| ivve | patch for the error, sorry :) | 12:47 |
| odyssey4me | well, if you can refer to the patch that needs merging that'd be helpful? | 12:48 |
| ivve | aye | 12:48 |
| *** spatel has quit IRC | 12:48 | |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible master: [WIP] Add qdrouterd role for messaging backend deployment https://review.openstack.org/570261 | 12:48 |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca master: Add Ubuntu Bionic 18.04 support https://review.openstack.org/586932 | 12:49 |
| evrardjp | jrosser: what's the 25% ? | 12:49 |
| evrardjp | is that updated on the etherpad? | 12:49 |
| ivve | https://bugs.launchpad.net/bugs/1783635 | 12:49 |
| openstack | Launchpad bug 1783635 in OpenStack Compute (nova) pike "Live migration failed.: AttributeError: 'Guest' object has no attribute 'migrate_configure_max_speed'" [High,Fix committed] - Assigned to Lee Yarwood (lyarwood) | 12:49 |
| ivve | it was just committed i see now | 12:49 |
| jrosser | evrardjp: designate, nova, octavia, trove, and a few minor roles that just look borked for * right now | 12:50 |
| evrardjp | jrosser: my bad we didn't have an etherpad for this. If it's the main roles, please change the template | 12:50 |
| evrardjp | jrosser: glance, cinder? | 12:50 |
| ivve | guess i need glasses | 12:50 |
| evrardjp | jrosser: ok so | 12:51 |
| jrosser | cinder is bust too sorry | 12:51 |
| evrardjp | if nova designate cinder are failing, I think it's not worth passing everything into voting. | 12:51 |
| jrosser | i have an etherpad for all this which i'll update | 12:51 |
| evrardjp | oh ok | 12:51 |
| odyssey4me | ivve: yeah, looks like that patch just, just merged - and our last bump on pike was a month ago | 12:51 |
| evrardjp | great | 12:51 |
| jrosser | i'd rather mark those roles specifically as nv | 12:51 |
| ivve | my git skills to get just that particular patch and back out later on are lacking :D | 12:51 |
| jrosser | than leave * else open to getting broken | 12:51 |
| odyssey4me | evrardjp: any idea when we're going to bump SHA's again? it looks like we're overdue? | 12:52 |
| evrardjp | ok. | 12:52 |
| *** pdeters has quit IRC | 12:52 | |
| evrardjp | odyssey4me: are we? | 12:52 |
| evrardjp | let me check the schedule | 12:52 |
| odyssey4me | oh hang, on - looking at the wrong branch | 12:52 |
| odyssey4me | apologies, got it wrong | 12:53 |
| evrardjp | odyssey4me: the last release took a while for the releases team to get in | 12:53 |
| odyssey4me | ivve was last bumped before that patch merged: https://github.com/openstack/openstack-ansible/blob/stable/pike/playbooks/defaults/repo_packages/openstack_services.yml#L150 | 12:53 |
| *** dave-mccowan has joined #openstack-ansible | 12:53 | |
| odyssey4me | if that patch is crucial, evrardjp could do an interim sha bump to pick it up so that it's in our next release | 12:53 |
| evrardjp | odyssey4me: you are right I missed the one on friday. | 12:54 |
| evrardjp | august a long month, which shifted the weeks, so to keep the two weeks schedule, releasing on the 3rd 17th AND 31th. | 12:55 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Add lxc3 compatibility https://review.openstack.org/589146 | 12:55 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [TEST] bionic https://review.openstack.org/586607 | 12:55 |
| *** ansmith has joined #openstack-ansible | 12:56 | |
| ansmith | odyssey4me: hello, quick question on mq setup, to add qdrouterd user setup, it should go here https://github.com/openstack/openstack-ansible-tests/blob/master/sync/tasks/donotuse_mq_setup.yml? | 12:57 |
| odyssey4me | ansmith: normally, yes - but I'm working through the roles right now to update them to work with tags, which is why that file name says 'donotuse' | 12:58 |
| ansmith | odyssey4me: should I wait until updates roll through? | 12:58 |
| *** shyambiradar has quit IRC | 12:59 | |
| odyssey4me | ansmith: the changes I'm making are shown in https://review.openstack.org/588190 - if you could do a patch on top of that one then I can combine it into my patches as I prep them across all the roles | 12:59 |
| jrosser | http://zuul.openstack.org/builds.html annoyingly auto-refreshes to not-the-thing-you-filtered-for :/ | 12:59 |
| *** dave-mccowan has quit IRC | 13:03 | |
| ivve | odyssey4me: would be great | 13:04 |
| ivve | when would it be merged otherwise, if not "escalated"? | 13:04 |
| Pradiprwt | Hi ivve, do you have some time I want your help, I am facing issue in installation of 3 node. The issue is with galera and haproxy communication. I don't have more idea about it | 13:07 |
| odyssey4me | Pradiprwt: Please don't harass everyone who communicates in this channel. In an open source community it is important to respect that people will help if they can, but will not if they cannot. You have received help and need to follow through with the advice you've been given already. | 13:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach master: Use generic vars file for ubuntu https://review.openstack.org/589074 | 13:09 |
| jrosser | here is my bionic etherpad for those interested in the current-ish state https://etherpad.openstack.org/p/osa-bionic | 13:15 |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca master: Upgrade zookeeper repository https://review.openstack.org/589151 | 13:17 |
| Pradiprwt | odyssey4me: Thanks for the suggestion, I am not trying to harass anyone.. It was request for help about unknown issue for me. I pinged those guys only who helped me erlier :( | 13:17 |
| *** jroll has quit IRC | 13:19 | |
| *** jroll has joined #openstack-ansible | 13:19 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Fix Bootstrap AIO Passing Multiple BOOTSTRAP_OPTS https://review.openstack.org/588839 | 13:19 |
| *** spatel has joined #openstack-ansible | 13:20 | |
| evrardjp | jrosser: I am fine with the global switch to voting, as long as it depends on patches introducing "non voting" for roles which break. | 13:21 |
| jrosser | evrardjp: ok, i'll start on sorting that out | 13:21 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: Update all SHAs for 17.0.9 https://review.openstack.org/589152 | 13:23 |
| *** hamzaachi_ has joined #openstack-ansible | 13:25 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Remove group vars hack https://review.openstack.org/584411 | 13:27 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: [WIP] Ensure python3-pkg-resources is installed https://review.openstack.org/566672 | 13:29 |
| *** Bhujay has quit IRC | 13:31 | |
| *** fghaas has joined #openstack-ansible | 13:32 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_zun master: Switch to using project-templates https://review.openstack.org/587536 | 13:34 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Switch to using project-templates https://review.openstack.org/586756 | 13:37 |
| fghaas | evrardjp: I'm really sorry that we need so many attempts at *actually* fixing that P-Q run-upgrade.sh issue, but I think I've finally figured out the real cause of the problem: it invokes the haproxy-install.yml playbook twice, setting the upgrade marker after the first run. When the upgrade script then attempts to run it again, it's just a no-op — but a no-op without --tags is fine, whereas one with --tags causes an error ( | 13:37 |
| fghaas | hence the initial red herring that we all fell for). Now clearly we do want to run haproxy-install.yml *after* the container cleanup. My question is, do you have any objections to removing the *prior* invocation of that same playbook, or does that have unintended consequences that you're aware of and I'm not? | 13:37 |
| odyssey4me | fghaas: you talking about removing this one? https://github.com/openstack/openstack-ansible/blob/stable/queens/scripts/run-upgrade.sh#L182 | 13:38 |
| *** olivierbourdon38 has quit IRC | 13:38 | |
| fghaas | Exactly, and leaving https://github.com/openstack/openstack-ansible/blob/stable/queens/scripts/run-upgrade.sh#L203 in. | 13:39 |
| odyssey4me | fghaas: removing that one would be a problem, because if any new openstack services come into play - and in this particular case they do (API services move to new or other containers) - the LB will not have the new endpoints configured, so things will fail during the install | 13:40 |
| *** olivierbourdon38 has joined #openstack-ansible | 13:41 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tempest master: Mark bionic gate jobs as non-voting https://review.openstack.org/589158 | 13:41 |
| *** MasterofJOKers has quit IRC | 13:41 | |
| fghaas | odyssey4me: OK, then there's the other option of removing the upgrade marker just before the second invocation of that same playbook to force it to re-run after the containers are cleaned up. Would that work? | 13:42 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.18 https://review.openstack.org/589159 | 13:42 |
| *** MasterofJOKers has joined #openstack-ansible | 13:43 | |
| odyssey4me | fghaas: oh, now I get what you're saying - the upgrade marker is making that second run get ignored | 13:43 |
| *** lbragstad has joined #openstack-ansible | 13:43 | |
| evrardjp | fghaas: what we did in the past is to have it different marker names | 13:44 |
| evrardjp | so that it ran twice | 13:44 |
| *** olivierbourdon38 has quit IRC | 13:44 | |
| *** olivierbourdon38 has joined #openstack-ansible | 13:44 | |
| odyssey4me | evrardjp: so just add some arbitrary CLI option so that the task is a different name? | 13:45 |
| evrardjp | odyssey4me: yeah so -e 'run_number=one' | 13:45 |
| odyssey4me | something like: RUN_TASKS+=("haproxy-install.yml -e foo=bar") | 13:45 |
| odyssey4me | ah yeah, that makes sense | 13:45 |
| evrardjp | you can even use it to explain things | 13:45 |
| odyssey4me | it's also very simple | 13:45 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_designate master: Mark bionic gate jobs as non-voting https://review.openstack.org/589160 | 13:46 |
| evrardjp | -e deploy_new_apis=True | 13:46 |
| evrardjp | or things like that | 13:46 |
| fghaas | Well that's what was previously tried with the `--tags` option and it never worked. | 13:46 |
| evrardjp | even if they don't mean anything | 13:46 |
| evrardjp | fghaas: yeah but tags mean something | 13:46 |
| evrardjp | where extra vars mean nothing if they aren't used | 13:46 |
| fghaas | Then there is a difference in how -e and --tags are handled? | 13:46 |
| evrardjp | and they are almost free, as they are not spread | 13:46 |
| *** olivierbourdon38 has quit IRC | 13:46 | |
| evrardjp | accross inventory | 13:46 |
| evrardjp | fghaas: so | 13:46 |
| *** olivierbourdon38 has joined #openstack-ansible | 13:47 | |
| evrardjp | basically this run upgrade runs an openstack-ansible cli, which is an ansible-playbook wrapper, as you know | 13:47 |
| fghaas | right | 13:47 |
| evrardjp | -e and --tags are not handled differently | 13:47 |
| evrardjp | they are just passed as cli | 13:47 |
| *** olivierbourdon38 has quit IRC | 13:47 | |
| evrardjp | but the upgrade marker is saving the entire call | 13:47 |
| evrardjp | so having two different lines will mean two runs | 13:48 |
| evrardjp | the difference here, is that -e 'babar=kingofelephants' is not used by haproxy play at all | 13:48 |
| evrardjp | where --tags haproxy-config really means something, and skips tasks | 13:48 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Mark bionic gate jobs as non-voting https://review.openstack.org/589161 | 13:49 |
| evrardjp | if a deployer runs things manually this issue wouldn't have appeared, as no upgrade marker would be considered | 13:49 |
| evrardjp | so I think it's fair to say it's a bug in run_upgrades.sh | 13:49 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: tasks: providers: Rename 'ovs' configuration file https://review.openstack.org/583919 | 13:49 |
| evrardjp | I prefer isolating what the plays are doing, so I think it's better to have two different markers for the two different plays, instead of removing existing marker (you don't have an easy information on when things ran if you remove the file) | 13:50 |
| evrardjp | I hope I have clarified things | 13:51 |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_horizon master: Implements custom theme distribution https://review.openstack.org/586318 | 13:51 |
| evrardjp | This is a good example of where a real CLI would shine. | 13:51 |
| *** spatel has quit IRC | 13:52 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: Mark bionic jobs as non-voting https://review.openstack.org/589162 | 13:52 |
| evrardjp | I think our salvation for fast deployer and easier upgrade is in python code we'll have to maintain with ansible strategies and a CLI. | 13:52 |
| evrardjp | but that's for later. | 13:52 |
| evrardjp | that should be completely optional so for your case, I think having just a different 'playbook call' would do the trick | 13:53 |
| evrardjp | I am curious fghaas I don't recall the upgrades, do we clean the conf.d folder? | 13:54 |
| evrardjp | else no cleanup would be done. | 13:54 |
| fghaas | evrardjp: sorry for sounding thick, but I don't quite follow. The cleanup-*.yml playbooks that run-upgrade.sh all get invoked with a bunch of -e options, but the upgrade markers they generate are just called /etc/openstack_deploy/upgrade-queens/cleanup-*.complete, so how would the markers differ between calls with various permutations of -e? | 13:54 |
| fghaas | (reference: https://github.com/openstack/openstack-ansible/blob/stable/queens/scripts/run-upgrade.sh#L64) | 13:55 |
| evrardjp | that's interesting fghaas I thought the full cli was written. | 13:55 |
| fghaas | nope. | 13:55 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Fix Bootstrap AIO Passing Multiple BOOTSTRAP_OPTS https://review.openstack.org/588840 | 13:55 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_masakari master: Mark bionic gate jobs as non-voting https://review.openstack.org/589165 | 13:56 |
| *** Guest70971 is now known as juhak | 13:56 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: Mark bionic gate jobs as non-voting https://review.openstack.org/589162 | 13:56 |
| evrardjp | fghaas: that's what was done in the leapfrogs: https://github.com/openstack/openstack-ansible-ops/blob/master/leap-upgrades/lib/functions.sh#L72-L80 | 13:57 |
| evrardjp | that's why I was confused, I thought we had the same thing. | 13:57 |
| *** mattt1 is now known as mattt | 13:58 | |
| evrardjp | I think we should have the full call. | 13:58 |
| evrardjp | ok so a few steps: | 13:58 |
| evrardjp | 1) Fix the script in master to introduce the full calls as markers, not only the playbook filename. | 13:59 |
| evrardjp | 2) backport it all the way | 13:59 |
| *** olivierbourdon38 has joined #openstack-ansible | 13:59 | |
| evrardjp | 3) For queens only, add an upgrade play that cleans up the conf.d/ folder | 13:59 |
| evrardjp | (For element 3) I suppose it's needed because I suppose we need to remove old cruft that would still be in haproxy, right?) | 14:00 |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca master: Add Ubuntu Bionic 18.04 support https://review.openstack.org/586932 | 14:01 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: Mark bionic gate jobs as non-voting https://review.openstack.org/589162 | 14:02 |
| fghaas | evrardjp: why not simply add a --force option to run_lock that disregards the presence of the upgrade marker? | 14:03 |
| fghaas | Also, not exactly sure why the conf.d cleanup is necessary. | 14:05 |
| evrardjp | fghaas: works for me too. But that would solve 1) | 14:05 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests master: Change bionic to be voting https://review.openstack.org/587478 | 14:05 |
| evrardjp | fghaas: let me check if we have implemented a cleanup of the conf.d folder in haproxy by default | 14:05 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests master: Change bionic to be voting https://review.openstack.org/587478 | 14:05 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests master: Change bionic to be voting https://review.openstack.org/587478 | 14:06 |
| openstackgerrit | Maksim Malchuk proposed openstack/openstack-ansible master: Leverage the environment variables for the deploy https://review.openstack.org/589167 | 14:06 |
| *** dxiri has joined #openstack-ansible | 14:07 | |
| *** dxiri has quit IRC | 14:07 | |
| *** dxiri has joined #openstack-ansible | 14:08 | |
| *** spotz has joined #openstack-ansible | 14:08 | |
| evrardjp | fghaas: the cleanup is only necessary if the regenerated files is a subset of the files present in conf.d | 14:09 |
| openstackgerrit | Maksim Malchuk proposed openstack/openstack-ansible master: Leverage the environment variables for the deploy https://review.openstack.org/589167 | 14:10 |
| fghaas | evrardjp: oh you mean /etc/haproxy/conf.d, as opposed to /etc/openstack_deploy/conf.d? | 14:10 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_panko master: [TEST] bionic https://review.openstack.org/589168 | 14:11 |
| evrardjp | fghaas: yes | 14:11 |
| evrardjp | so | 14:11 |
| fghaas | Why does that need to be separately cleaned out, if we're already running the full haproxy-install.yml playbook? | 14:12 |
| evrardjp | assuming a service was removed (IIRC it was queens, or maybe rocky,I can't remember), it would need removing the files | 14:12 |
| evrardjp | fghaas: so | 14:12 |
| evrardjp | let's check this with real data | 14:12 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Allow tags to be used for MQ tasks https://review.openstack.org/588190 | 14:15 |
| evrardjp | fghaas: ok so, between pike and queens, no removed service. So all I said for 3) doesn't matter. | 14:15 |
| evrardjp | It does matter for R though. | 14:16 |
| fghaas | hmmm, if so, shouldn't that be part of the haproxy-install.yml playbook itself, as opposed to an upgrade playbook? | 14:17 |
| evrardjp | the reasoning is that if a service is removed, and file for it is not removed from /etc/haproxy/conf.d/ then the file will still be there on an upgrade, causing the backend/frontend to still be there. | 14:17 |
| evrardjp | fghaas: we would need to store a fact of previous run, to know if things are legit or not | 14:17 |
| *** Bhujay has joined #openstack-ansible | 14:18 | |
| evrardjp | if you don't, and cleanup the files all the time, then you're losing important idempotency data. | 14:18 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift master: Allow tags to be used for MQ tasks https://review.openstack.org/588196 | 14:21 |
| evrardjp | maybe we can have both at the same time, by ensuring all the things are always rightly sorted, and marking the removal as changed_when: false | 14:21 |
| fghaas | I wasn't suggesting to empty that directory on every run. I was thinking out loud that the role could perhaps nuke any *unmanaged* files in that directory. | 14:21 |
| evrardjp | we jsut didn't have the need for now. | 14:21 |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 14:22 |
| openstackgerrit | Maksim Malchuk proposed openstack/openstack-ansible stable/queens: Leverage the environment variables for the deploy https://review.openstack.org/589176 | 14:22 |
| evrardjp | oh interesting concept. Removing all the files from a find which don't belong to existing set. | 14:22 |
| evrardjp | that's defintely easier to do than before. We should do this! | 14:23 |
| *** kklimonda_ has joined #openstack-ansible | 14:23 | |
| noonedeadpunk | odyssey4me: it's strange, that tests fail for masakari, as container are created without any problems on my test environment... | 14:23 |
| fghaas | right, do note however that hic sunt dragones because ansible sometimes makes weird assumptions about file globs | 14:23 |
| *** Tahvok_ has joined #openstack-ansible | 14:23 | |
| evrardjp | definitely better than the change_when: false, which breaks a few expectations. | 14:23 |
| evrardjp | fghaas: I am not sure about what you meant there. | 14:25 |
| evrardjp | I will also completely forget this except if we put this in a bug. | 14:25 |
| hwoarang | so | 14:25 |
| hwoarang | final integration patch for distro installation it seems https://review.openstack.org/#/c/584765/ | 14:26 |
| * hwoarang needs to write some docs now | 14:26 | |
| *** kklimonda has quit IRC | 14:26 | |
| *** Tahvok has quit IRC | 14:26 | |
| *** toddnni has quit IRC | 14:26 | |
| *** dankolbrs has quit IRC | 14:26 | |
| *** coolj has quit IRC | 14:26 | |
| *** flaviosr has quit IRC | 14:26 | |
| *** csmart[m] has quit IRC | 14:26 | |
| *** jothibasu[m] has quit IRC | 14:26 | |
| *** marst has quit IRC | 14:26 | |
| *** hw_wutianwei_ has quit IRC | 14:26 | |
| *** szaher has quit IRC | 14:26 | |
| *** bjolo has quit IRC | 14:26 | |
| *** Tahvok_ is now known as Tahvok | 14:26 | |
| *** kklimonda_ is now known as kklimonda | 14:26 | |
| d34dh0r53 | S P L I T | 14:26 |
| hwoarang | 1h48m is not a really good time though | 14:26 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.26 https://review.openstack.org/589178 | 14:27 |
| *** toddnni has joined #openstack-ansible | 14:27 | |
| fghaas | evrardjp: well you can't do state: absent with file and expect to be able to use a shell glob. And you also can't do with_fileglob because that globs locally (i.e. on your deploy host) and not on the target box. | 14:27 |
| fghaas | So yes, it would be a dance involving find and register variables | 14:28 |
| evrardjp | yes ofc. But the glob of the find module should work | 14:28 |
| evrardjp | we need to use this, because we need to explictly do a difference of the two sets of files anyway : ) | 14:28 |
| *** marst has joined #openstack-ansible | 14:29 | |
| *** Bhujay has quit IRC | 14:29 | |
| fghaas | evrardjp: trouble with Queens is that we currently bootstrap 2.4, and the find module addes excludes in 2.5 | 14:29 |
| *** mmalchuk has quit IRC | 14:29 | |
| fghaas | added | 14:29 |
| evrardjp | well in any case P->Q is fine by default | 14:30 |
| *** galaxyblr has quit IRC | 14:30 | |
| evrardjp | as there is no service removed | 14:30 |
| *** flaviosr has joined #openstack-ansible | 14:30 | |
| evrardjp | but Q->R needs something. | 14:30 |
| fghaas | ah, right. | 14:30 |
| evrardjp | I meant a difference filter anyway, so it's fine. | 14:30 |
| evrardjp | nothing undoable | 14:30 |
| evrardjp | we have a find module now, and that's the only thing that's important here : ) | 14:31 |
| kaiokmo | can I get a review on those? https://goo.gl/2khBSG | 14:33 |
| fghaas | evrardjp: so list all files in that directory, apply difference filter with known good filenames, and then do state: absent on file with with_items from that list. | 14:33 |
| evrardjp | indeed. | 14:33 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift master: vars: Add missing python-keystonemiddleware dependency https://review.openstack.org/589128 | 14:34 |
| fghaas | good thing about that is it can land in master now and not need a backport | 14:34 |
| evrardjp | oh wait. | 14:34 |
| evrardjp | TILT. | 14:34 |
| evrardjp | https://github.com/openstack/openstack-ansible/commit/14d4da73358f3fc92a0191ce1c14d566030d033c | 14:34 |
| evrardjp | I am sorry. But we found a better way to reduce tech debt though. | 14:35 |
| evrardjp | anyway, yeah the force options for your use case is probably the simpler, it can be done in master and backported easily, and would need no much change. | 14:36 |
| fghaas | So you're telling me it's already doing what you want it to do, just not as elegantly as it could be done? | 14:36 |
| fghaas | (with that patch you just highlighted, I mean) | 14:37 |
| evrardjp | yeah. Sorry I am sorry for the time loss, I already thought of this issue 6 months ago, I just didn't remember. | 14:39 |
| fghaas | ah no worries at all, happens to the best of us :) | 14:39 |
| evrardjp | I am now willing to improve this further in the future. Let me file a bug. | 14:40 |
| jrosser | kaiokmo: done! i'll keep an eye on whats going on with bionic there | 14:43 |
| *** eglute has quit IRC | 14:45 | |
| kaiokmo | thanks, jrosser | 14:46 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Remove octavia_requires_pip_packages https://review.openstack.org/588177 | 14:48 |
| evrardjp | hwoarang: it only passes for suse though :( | 14:49 |
| hwoarang | evrardjp: yes like most of the roles :) | 14:49 |
| evrardjp | hahah | 14:49 |
| evrardjp | I see | 14:49 |
| hwoarang | (same deal with distro_basekit) | 14:49 |
| hwoarang | centos7 needs some love and xenial we said we can't do distro work because they dont provide packages | 14:50 |
| hwoarang | hopefully once jrosser finishes with bionic we can drop xenial distro jobs and enable bionic | 14:50 |
| evrardjp | hwoarang: yeah makes sense. | 14:51 |
| evrardjp | I'd deploy everything on metal if it's too slow | 14:51 |
| *** gyee has joined #openstack-ansible | 14:52 | |
| jrosser | what is going on here? https://review.openstack.org/#/c/584609/ | 14:52 |
| jrosser | "http://mirror.mtl01.inap.openstack.org/ubuntu-cloud-archive bionic-updates/queens Release' does not have a Release file" | 14:53 |
| evrardjp | I think it's time we rethink the jobs -- basekit + metal + sources // basekit + metal + distro pkg // basekit + lxc/nspawn + sources | 14:53 |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca-agent master: Add support for openSUSE https://review.openstack.org/588462 | 14:53 |
| *** spatel has joined #openstack-ansible | 14:53 | |
| jrosser | why is that job even trying to use a queens repo on a bionic install | 14:54 |
| evrardjp | // integrated/translations + lxc + sources | 14:54 |
| spatel | cloudnull: now my horizon running blazing speed... after fixing that kernel/printk stuff | 14:54 |
| evrardjp | spatel: what was the issue? | 14:54 |
| spatel | evrardjp: here is the details i put https://bugs.launchpad.net/openstack-ansible/+bug/1785365 | 14:55 |
| openstack | Launchpad bug 1785365 in openstack-ansible "LXC container network slow because of kernel debug mesg" [Undecided,New] | 14:55 |
| spatel | "echo "3 4 1 3" > /proc/sys/kernel/printk" this was fix | 14:55 |
| spatel | before printk was set to "7 4 1 3" 7 - Debug mode so it was flooding kernel stack error generated by iptables checksum-fill option https://bugs.launchpad.net/openstack-ansible/+bug/1785517 | 14:56 |
| openstack | Launchpad bug 1785517 in openstack-ansible "iptables checksum-fill causing kernel warning error stack" [Undecided,New] | 14:56 |
| evrardjp | spotz: oh bad offload? | 14:57 |
| spatel | In short iptables checksum-fill was sending tons shit of logs to kernel and kernel was running in debug mode | 14:57 |
| *** fghaas has quit IRC | 14:57 | |
| spotz | evrardjp: what? | 14:57 |
| *** dave-mccowan has joined #openstack-ansible | 14:57 | |
| spatel | because of that flooding my all lxc container was running poor network speed | 14:57 |
| evrardjp | spotz: sorry it was for spatel | 14:57 |
| spotz | ahh | 14:57 |
| evrardjp | spatel: interesting, did you check what's the current value in gates by any chance? | 14:58 |
| spatel | gates ? | 14:58 |
| spatel | long story short this rules was causing all issue "-A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill" | 14:59 |
| spatel | checksum-fill was generating kernel error stack | 14:59 |
| *** Neptu_ is now known as Neptu | 14:59 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [DNM] test values on some infra providers https://review.openstack.org/589183 | 14:59 |
| evrardjp | spatel: understood. | 14:59 |
| spatel | evrardjp: what you asking about gates value? | 15:00 |
| evrardjp | I will check if by any chance, we could have the same thing going on in gates | 15:00 |
| evrardjp | see https://review.openstack.org/589183 | 15:00 |
| evrardjp | this will always fail but output the value | 15:00 |
| evrardjp | if I see a provider that have the same thing, it means we could have a way to accelerate gates. | 15:00 |
| spatel | In my repo i don't have that "cat /proc/sys/kernel/printk" line | 15:02 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [DNM] test values on some infra providers https://review.openstack.org/589183 | 15:02 |
| evrardjp | spatel: nevermind | 15:02 |
| spatel | This is what i set in my user_variable to fix this issue | 15:02 |
| -spatel- openstack_user_kernel_options: | 15:03 | |
| -spatel- - { key: 'kernel.printk', value: 3 4 1 3 } | 15:03 | |
| hwoarang | jrosser: i think it's because of the upgrade | 15:03 |
| jrosser | oh! | 15:03 |
| hwoarang | old galera is being deployed, pulling old openstack_hosts, using queens as repo | 15:03 |
| jrosser | no upgrade in the job name though? | 15:03 |
| hwoarang | i just see this line http://logs.openstack.org/09/584609/1/gate/openstack-ansible-functional-ubuntu-bionic-ssl/baeb38e/job-output.txt.gz#_2018-08-01_14_33_15_619919 | 15:03 |
| spatel | evrardjp: i would say we should push this change to make prink run in (no-debug) | 15:04 |
| *** vnogin has quit IRC | 15:04 | |
| hwoarang | so i am just guessing :) | 15:04 |
| spatel | i believe all CentOS7 users must be having this issue.. I can't speak for ubuntu | 15:04 |
| evrardjp | jrosser: hwoarang it's fine to clone the role, it doesn't seem to use it | 15:05 |
| evrardjp | jrosser: however, I'd like to make sure we don't RUN old roles : ) | 15:05 |
| evrardjp | it doesn't seem so | 15:05 |
| evrardjp | but loading variables from it is something to avoid | 15:05 |
| jrosser | something certainly has got hold of a queens repo there | 15:05 |
| evrardjp | jrosser: is there somehting you see in the gate logs? | 15:05 |
| *** luksky has quit IRC | 15:05 | |
| evrardjp | we dump the content of apt repos somewhere | 15:05 |
| jrosser | what hwoarang linked up there ^^ | 15:06 |
| jrosser | oh i see what you mean | 15:06 |
| hwoarang | also this | 15:06 |
| hwoarang | http://logs.openstack.org/09/584609/1/gate/openstack-ansible-functional-ubuntu-bionic-ssl/baeb38e/job-output.txt.gz#_2018-08-01_14_35_22_013533 | 15:06 |
| hwoarang | loading vars/ubuntu.yml | 15:06 |
| hwoarang | there is no such file anymore | 15:06 |
| *** DanyC has quit IRC | 15:10 | |
| jrosser | there was only a very short window when vars/ubuntu.yml was a thing - it was wrong to change it | 15:11 |
| *** pcaruana has quit IRC | 15:12 | |
| *** cshen has quit IRC | 15:14 | |
| *** shyambiradar has joined #openstack-ansible | 15:16 | |
| *** rmart04 has quit IRC | 15:17 | |
| *** vnogin has joined #openstack-ansible | 15:17 | |
| *** FrankZhang has joined #openstack-ansible | 15:17 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_monasca-agent master: Switch to using project-templates https://review.openstack.org/589189 | 15:18 |
| *** mmalchuk has joined #openstack-ansible | 15:22 | |
| *** spatel has quit IRC | 15:23 | |
| *** ebbex has joined #openstack-ansible | 15:23 | |
| *** olivierbourdon38 has quit IRC | 15:24 | |
| *** mmalchuk_ has joined #openstack-ansible | 15:25 | |
| *** mmalchuk has quit IRC | 15:26 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_monasca master: Switch to using project-templates https://review.openstack.org/587533 | 15:29 |
| d34dh0r53 | odyssey4me: re that swift thing: http://paste.openstack.org/show/727421/ | 15:29 |
| d34dh0r53 | I checked the facts in /etc/openstack_deploy/ansible_facts and that key does not exist, the md5sum command that should register that fact works fine so for some reason ansible is failing to register a fact | 15:29 |
| *** vnogin has quit IRC | 15:30 | |
| *** vnogin has joined #openstack-ansible | 15:30 | |
| *** shyam89 has joined #openstack-ansible | 15:31 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests master: Change bionic to be voting https://review.openstack.org/587478 | 15:32 |
| jrosser | my bionic aio was still going at 2hr point when it timed out..... slooooow | 15:34 |
| *** shyambiradar has quit IRC | 15:35 | |
| *** throwsb1 has joined #openstack-ansible | 15:36 | |
| evrardjp | jrosser: with lxc? | 15:42 |
| jrosser | yeah | 15:43 |
| jrosser | evrardjp: heres my hacky test of bionic aio https://review.openstack.org/#/c/586607/ | 15:43 |
| *** Leo_m has joined #openstack-ansible | 15:43 | |
| evrardjp | spatel BINGO! | 15:43 |
| odyssey4me | d34dh0r53: my personal view is that os_swift could do with a pretty serious overhaul - it relies far too much on the fact cache and on delegation, but it'd be a lot of work for little gain given that most people seem to be using ceph and ceph-rgw instead | 15:43 |
| evrardjp | mnaser: wooot we found out a way to help the speed of centos! | 15:44 |
| evrardjp | http://logs.openstack.org/83/589183/2/check/openstack-ansible-deploy-aio_metal-centos-7/55ca0c3/job-output.txt.gz#_2018-08-06_15_19_44_745924 | 15:44 |
| evrardjp | (maybe) | 15:45 |
| mnaser | evrardjp: i don't follow, high level of logging slows things down? | 15:45 |
| odyssey4me | d34dh0r53: I guess that doesn't help you much if Swift is important to a stable branch. | 15:45 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_monasca master: Switch to using project-templates https://review.openstack.org/587533 | 15:45 |
| evrardjp | mnaser: yes. | 15:45 |
| evrardjp | it's only set like this for centos -- at least on ovh | 15:45 |
| evrardjp | I am rechecking to see. | 15:46 |
| evrardjp | But right now OVH is also running on qemu without kvm | 15:46 |
| evrardjp | so these things are piling up | 15:46 |
| bgmccollum | can we blacklist a provider? | 15:46 |
| evrardjp | nope | 15:46 |
| mnaser | nope | 15:46 |
| mnaser | jinks | 15:46 |
| mnaser | :p | 15:46 |
| evrardjp | haha | 15:46 |
| evrardjp | it is jinks or jinx ? | 15:47 |
| mnaser | i dunno, probably the same | 15:47 |
| bgmccollum | can we flip a coin on a blacklisted provider, and allow or deny? :P | 15:47 |
| odyssey4me | djinks | 15:47 |
| odyssey4me | :p | 15:47 |
| bgmccollum | call it a greylist | 15:47 |
| evrardjp | odyssey4me: you so ZA | 15:48 |
| evrardjp | bgmccollum: that's a lot of rechecks. | 15:48 |
| d34dh0r53 | odyssey4me: yeah, what is really interesting to me is that this seems to be a transient issue but it happens every time if I include my MNAIO ELK stuff, I'm thinking that it's an ansible bug since set_fact really should do what it says. I agree with you WRT swift though, it needs a lot of work and that work may not even be worth it at this point | 15:48 |
| odyssey4me | d34dh0r53: perhaps it's just a simple storage race condition - ansible's reading the fact before it's been flushed to disk, and this is happening due to resource pressure | 15:49 |
| d34dh0r53 | hmm, interesting, I wonder if I can force a flush | 15:49 |
| bgmccollum | but now that i think about it, qemu shouldn't affect much of anything...cirros on qemu isn't terrible | 15:49 |
| bgmccollum | and thats what tempest uses...so... | 15:49 |
| evrardjp | bgmccollum: yeah but we get lots of tempest issues of timeouts | 15:50 |
| odyssey4me | d34dh0r53: you could try disabling the fact cache so that only memory is used - or use memcache for the fact cache | 15:50 |
| evrardjp | I made the timeouts twice as high as standard defaults, reducing the amount of timeout issues quite radically. But we still hit those. | 15:50 |
| *** shyambiradar has joined #openstack-ansible | 15:51 | |
| *** shyam89 has quit IRC | 15:55 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Switch to using project-templates https://review.openstack.org/586756 | 15:56 |
| jrosser | could i get reviews on the passing jobs here? https://review.openstack.org/#/q/topic:bionic-check+(status:open) | 16:03 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-tests master: Revert "Run with KVM if we can in tests" https://review.openstack.org/589207 | 16:03 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Revert "Work around nested virt issues on OVH test nodes" https://review.openstack.org/589209 | 16:04 |
| logan- | evrardjp: we should be clear to turn on kvm nested virt in OVH again. johnsom worked with them and tracked down the problem, it got resolved about a month ago: https://review.openstack.org/#/c/579215/ | 16:05 |
| johnsom | Yeah, we have re-enabled kvm on all providers in the Octavia gates | 16:05 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_monasca-agent master: Switch to using project-templates https://review.openstack.org/589189 | 16:06 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_monasca-agent master: Switch to using project-templates https://review.openstack.org/589189 | 16:07 |
| evrardjp | ok | 16:07 |
| evrardjp | did we do it in OSA? | 16:07 |
| logan- | working on it | 16:08 |
| logan- | https://review.openstack.org/#/c/589207/ and https://review.openstack.org/#/c/589209/ | 16:08 |
| evrardjp | doesn't look like it | 16:08 |
| evrardjp | cool | 16:08 |
| evrardjp | thanks | 16:08 |
| evrardjp | oh they are in merge conflicts | 16:08 |
| evrardjp | logan-: are you fixing those or should I do new patches or rebase properly | 16:09 |
| logan- | im rebasing them right now | 16:09 |
| evrardjp | ok I had the fix | 16:11 |
| evrardjp | I will leave that to you. | 16:11 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-tests master: Revert "Run with KVM if we can in tests" https://review.openstack.org/589207 | 16:12 |
| *** fghaas has joined #openstack-ansible | 16:12 | |
| evrardjp | logan-: just edit on a new thing, use the same commit message should be faster, small tip : ) | 16:12 |
| logan- | yeah it would have been much faster that way lol | 16:13 |
| *** suggestable has quit IRC | 16:13 | |
| evrardjp | I had my patches I stopped the git-review because you said so : ) | 16:13 |
| evrardjp | hahaha | 16:13 |
| evrardjp | anyway | 16:13 |
| evrardjp | It's cool | 16:13 |
| *** fghaas has quit IRC | 16:13 | |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Revert "Work around nested virt issues on OVH test nodes" https://review.openstack.org/589209 | 16:16 |
| evrardjp | folks I am merging this very quick ^ | 16:17 |
| FrankZhang | odyssey4me hey man, how's it going? Just wanna quick check with you about neutron project group relocating card: https://review.openstack.org/#/c/586269/ | 16:17 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_congress master: Use generic vars file for ubuntu https://review.openstack.org/589215 | 16:20 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_congress master: Switch to using project-templates https://review.openstack.org/587532 | 16:23 |
| *** shyambiradar has quit IRC | 16:26 | |
| *** bgmccollum has left #openstack-ansible | 16:27 | |
| *** shyambiradar has joined #openstack-ansible | 16:27 | |
| *** hamzaachi_ has quit IRC | 16:27 | |
| *** devx has joined #openstack-ansible | 16:30 | |
| devx | cloudnull or jrosser have any of you ran into a kibana mapping conflict? | 16:30 |
| jrosser | devx: yes! but as yet i've not really know what to do about it | 16:32 |
| devx | ok, I'll look in to it. Wanted to ask before I went down the rabbit hole | 16:32 |
| jrosser | i didnt even manage to find which things were in conflict - slightly surprising it doesnt list them for you | 16:33 |
| devx | no it just gave me a generic message | 16:33 |
| devx | OK, it was an easy fix. | 16:35 |
| devx | for me it was set the "read_only_allow_delete": "false" and then run the createElasticIndexes.yml | 16:36 |
| jrosser | oh my i really really need to fix this retention period :( | 16:36 |
| *** bgmccollum has joined #openstack-ansible | 16:37 | |
| *** SamYaple has quit IRC | 16:37 | |
| *** SamYaple has joined #openstack-ansible | 16:38 | |
| devx | when i ran the create index playbook it gave me this error '[FORBIDDEN/12/index read-only / allow delete (api)]' | 16:38 |
| *** vnogin has quit IRC | 16:41 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_monasca-agent master: Switch to using project-templates https://review.openstack.org/589189 | 16:42 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_watcher master: Switch to using project-templates https://review.openstack.org/587534 | 16:44 |
| *** DanyC has joined #openstack-ansible | 16:46 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_searchlight master: Switch to using project-templates https://review.openstack.org/587541 | 16:48 |
| *** evrardjp has quit IRC | 16:51 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_magnum master: Switch to using project-templates https://review.openstack.org/588380 | 16:51 |
| openstackgerrit | Maksim Malchuk proposed openstack/openstack-ansible stable/queens: Leverage the environment variables for the deploy https://review.openstack.org/589176 | 16:53 |
| *** olivierbourdon38 has joined #openstack-ansible | 16:56 | |
| *** luksky has joined #openstack-ansible | 16:59 | |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_cinder master: Add qdrouterd user to mq setup https://review.openstack.org/589234 | 17:03 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_cinder master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/582632 | 17:08 |
| *** rmart04 has joined #openstack-ansible | 17:11 | |
| odyssey4me | FrankZhang: apologies not yet - that didn't work, and nor did my other thing, so I need to spend a bit more time on it - or someone else with more time does... I'm trying to close off a bunch of work before the Rocky RC and while your patch would be nice, it's not adding any new function, just a new test, so it seems lower priority to me | 17:19 |
| odyssey4me | xgerman_ johnsom I think I made some mistakes. If you guys have some time to validate some things I'd like to confirm before I go and fix them up again. | 17:20 |
| *** ppetit has quit IRC | 17:21 | |
| *** savvas has joined #openstack-ansible | 17:24 | |
| noonedeadpunk | odyssey4me: do you have any guesses why masakari role fails functional test on lxc_create role? It seems strange, that it fails there, as there's no reason in role, why it couldn't access container after creation... Another question, that installing only nova-compute will fail on this task https://github.com/openstack/openstack-ansible-os_nova/blob/b18668360c88abb100b2d223c00fcda97dfd4b4c/tasks/nova_compute_wait.yml#L24-L30 | 17:28 |
| odyssey4me | noonedeadpunk: yeah, it is odd - likely an inventory issue, but haven't had a chance to figure it out yet | 17:28 |
| odyssey4me | noonedeadpunk: doing only nova compute alone was a chance, I guess it'll need more nova services - at the very least the conductor service | 17:29 |
| noonedeadpunk | odyssey4me: it runs ok on my test server (just running ./run_test.sh on Ubuntu/Centos) - it fails only on this step... | 17:29 |
| noonedeadpunk | odyssey4me: yep, without conductor it fails on including this file from main.yml | 17:30 |
| noonedeadpunk | is there more correct way to reproduce zuul environment on some sandbox? | 17:33 |
| *** fghaas has joined #openstack-ansible | 17:33 | |
| *** shyambiradar has quit IRC | 17:36 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Ensure the octavia flavor is private https://review.openstack.org/589239 | 17:37 |
| odyssey4me | noonedeadpunk: yes, make sure you start the test using 'run_tests.sh functional', but start it using a non-root user which has sudo capabilities | 17:38 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Ensure the octavia flavor is private https://review.openstack.org/589239 | 17:39 |
| *** olivierbourdon38 has quit IRC | 17:40 | |
| *** evrardjp has joined #openstack-ansible | 17:41 | |
| *** vnogin has joined #openstack-ansible | 17:42 | |
| noonedeadpunk | odyssey4me I used to run it from root user, so I'll try to run with non-root user, thanks | 17:42 |
| *** fghaas has quit IRC | 17:42 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Ensure the octavia network setup is in the right project https://review.openstack.org/589240 | 17:44 |
| odyssey4me | noonedeadpunk: yeah, it's an unfortunate confusion - and not very obvious... we'll hopefully get to fix that some time soon | 17:44 |
| *** hamzaachi has joined #openstack-ansible | 17:49 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Add retries to service role creation https://review.openstack.org/589245 | 17:56 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Use interface instead of endpoint_type https://review.openstack.org/589246 | 17:56 |
| johnsom | odyssey4me What can I help you with? | 17:57 |
| odyssey4me | johnsom: well, I kind realised that the image, flavor, networks, etc all needed to be put into the octavia project, not the admin project - and that the ansible modules don't let you do that cross-project | 17:58 |
| odyssey4me | johnsom: so I've pushed up the above patches to os_octavia to reverse some of the changes to correct that | 17:59 |
| odyssey4me | I'd appreciate validation that my suspicion is right. :) | 17:59 |
| johnsom | Ok, yeah, those are owned by the octavia service account | 17:59 |
| johnsom | project service, username octavia | 17:59 |
| odyssey4me | Yeah, thought as much. So I broke things - but the patches are up to fix them again. | 17:59 |
| johnsom | Ok, will have a look | 18:00 |
| *** poopcat has joined #openstack-ansible | 18:02 | |
| odyssey4me | johnsom: thanks, appreciate it - apologies for the breakage | 18:02 |
| *** gillesMo has quit IRC | 18:05 | |
| *** cshen has joined #openstack-ansible | 18:10 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Make role test-requirements.txt be additive https://review.openstack.org/589248 | 18:14 |
| *** cshen has quit IRC | 18:15 | |
| *** vnogin has quit IRC | 18:23 | |
| *** vnogin has joined #openstack-ansible | 18:23 | |
| *** nicolasbock has quit IRC | 18:23 | |
| *** nicolasbock has joined #openstack-ansible | 18:24 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Revert "Run with KVM if we can in tests" https://review.openstack.org/589207 | 18:27 |
| *** cshen has joined #openstack-ansible | 18:29 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Use repo_build_pip_default_index for the pypi fallback https://review.openstack.org/587871 | 18:31 |
| *** hamzaachi has quit IRC | 18:32 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Clean up tests a bit https://review.openstack.org/589259 | 18:32 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-tests stable/queens: Revert "Run with KVM if we can in tests" https://review.openstack.org/589261 | 18:34 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-tests stable/pike: Revert "Run with KVM if we can in tests" https://review.openstack.org/589262 | 18:35 |
| *** vnogin has quit IRC | 18:36 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Clean up tests a bit https://review.openstack.org/589259 | 18:37 |
| *** DanyC has quit IRC | 18:37 | |
| openstackgerrit | Maksim Malchuk proposed openstack/openstack-ansible stable/queens: Leverage the environment variables for the deploy https://review.openstack.org/589176 | 18:39 |
| *** monty has quit IRC | 18:40 | |
| *** monty has joined #openstack-ansible | 18:41 | |
| *** vnogin has joined #openstack-ansible | 18:42 | |
| *** fghaas has joined #openstack-ansible | 18:47 | |
| jrosser | cores: can i get reviews on these please https://review.openstack.org/#/q/topic:bionic-check+(status:open) ? | 18:47 |
| jrosser | i need to get those in before i can switch the bionic roles to be voting, they're already regressing in places :/ | 18:47 |
| *** fghaas has quit IRC | 18:49 | |
| *** kaiokmo has quit IRC | 18:58 | |
| *** rmart04 has quit IRC | 18:59 | |
| *** goldenfri has joined #openstack-ansible | 19:24 | |
| *** hamzaachi has joined #openstack-ansible | 19:27 | |
| *** dariko has joined #openstack-ansible | 19:28 | |
| *** hamzaachi has quit IRC | 19:33 | |
| *** dariko has quit IRC | 19:33 | |
| *** kaiokmo has joined #openstack-ansible | 19:34 | |
| *** vnogin has quit IRC | 19:36 | |
| *** aetaric has quit IRC | 19:38 | |
| *** goldenfri has quit IRC | 19:46 | |
| *** fghaas has joined #openstack-ansible | 19:47 | |
| logan- | jrosser: mind if i chop the stray whitespace off of https://review.openstack.org/#/c/589160 | 19:51 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-os_designate master: Mark bionic gate jobs as non-voting https://review.openstack.org/589160 | 19:55 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-os_almanach master: Switch to using project-templates https://review.openstack.org/587531 | 19:55 |
| *** zul has quit IRC | 19:59 | |
| openstackgerrit | Antony Messerli proposed openstack/openstack-ansible-ops master: Remove destroy reminent grouped containers https://review.openstack.org/589272 | 19:59 |
| *** cshen has quit IRC | 20:02 | |
| *** fghaas has quit IRC | 20:02 | |
| jrosser | logan-: go for it! | 20:07 |
| *** hamzaachi has joined #openstack-ansible | 20:07 | |
| openstackgerrit | Antony Messerli proposed openstack/openstack-ansible-ops master: Remove hardening from leap upgrades https://review.openstack.org/589275 | 20:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca-agent master: Switch to using project-templates https://review.openstack.org/589189 | 20:15 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher master: Switch to using project-templates https://review.openstack.org/587534 | 20:21 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_congress master: Switch to using project-templates https://review.openstack.org/587532 | 20:22 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Switch to using project-templates https://review.openstack.org/586756 | 20:26 |
| openstackgerrit | Merged openstack/openstack-ansible master: Revert "Work around nested virt issues on OVH test nodes" https://review.openstack.org/589209 | 20:27 |
| openstackgerrit | Merged openstack/openstack-ansible-os_searchlight master: Switch to using project-templates https://review.openstack.org/587541 | 20:27 |
| *** ansmith has quit IRC | 20:29 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_zaqar master: Switch to using project-templates https://review.openstack.org/587535 | 20:33 |
| *** vnogin has joined #openstack-ansible | 20:36 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_zun master: Switch to using project-templates https://review.openstack.org/587536 | 20:49 |
| *** FrankZhang has quit IRC | 21:00 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Mark bionic gate jobs as non-voting https://review.openstack.org/589162 | 21:12 |
| *** dxiri has quit IRC | 21:22 | |
| *** hwoarang has quit IRC | 21:24 | |
| *** dxiri has joined #openstack-ansible | 21:25 | |
| *** DanyC has joined #openstack-ansible | 21:26 | |
| *** DanyC has quit IRC | 21:37 | |
| *** hamzaachi has quit IRC | 21:46 | |
| *** ansmith has joined #openstack-ansible | 21:47 | |
| *** throwsb1 has quit IRC | 22:04 | |
| *** mmalchuk_ has quit IRC | 22:06 | |
| *** mmalchuk has joined #openstack-ansible | 22:07 | |
| *** mmalchuk has quit IRC | 22:11 | |
| *** mmalchuk has joined #openstack-ansible | 22:18 | |
| *** mmalchuk has quit IRC | 22:22 | |
| *** nicolasbock has quit IRC | 22:25 | |
| *** luksky has quit IRC | 22:25 | |
| *** mmalchuk has joined #openstack-ansible | 22:35 | |
| *** mmalchuk has quit IRC | 22:40 | |
| *** mmalchuk has joined #openstack-ansible | 22:48 | |
| *** mmalchuk has quit IRC | 22:53 | |
| *** guilhermesp_ has joined #openstack-ansible | 22:56 | |
| *** guilhermesp_ has quit IRC | 22:56 | |
| *** Leo_m has quit IRC | 22:57 | |
| *** mmalchuk has joined #openstack-ansible | 23:01 | |
| *** tosky has quit IRC | 23:02 | |
| *** mmalchuk has quit IRC | 23:06 | |
| *** mmalchuk has joined #openstack-ansible | 23:18 | |
| *** mmalchuk has quit IRC | 23:23 | |
| *** mmalchuk has joined #openstack-ansible | 23:35 | |
| *** mmalchuk has quit IRC | 23:40 | |
| *** mmalchuk has joined #openstack-ansible | 23:52 | |
| *** mmalchuk has quit IRC | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!