| *** dsneddon has quit IRC | 00:01 | |
| *** dsneddon has joined #oooq | 00:12 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container- (2 more messages) | 00:31 |
|---|---|---|
| *** rlandy has quit IRC | 00:37 | |
| *** kopecmartin|off has quit IRC | 01:50 | |
| *** kopecmartin has joined #oooq | 01:51 | |
| *** jrist- has joined #oooq | 01:53 | |
| *** kopecmartin has quit IRC | 01:57 | |
| *** kopecmartin has joined #oooq | 01:59 | |
| *** jbadiapa has quit IRC | 02:00 | |
| *** apetrich has quit IRC | 02:00 | |
| *** marios has quit IRC | 02:00 | |
| *** sshnaidm has quit IRC | 02:00 | |
| *** jrist has quit IRC | 02:00 | |
| *** matbu has quit IRC | 02:00 | |
| *** sshnaidm has joined #oooq | 02:05 | |
| *** apetrich has joined #oooq | 02:06 | |
| *** jbadiapa has joined #oooq | 02:07 | |
| *** kopecmartin has quit IRC | 02:09 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container- (2 more messages) | 02:31 |
| *** hamzy has joined #oooq | 02:33 | |
| jrist- | anybody still on? weshay? | 02:53 |
| jrist- | got a real-funny error | 02:53 |
| jrist- | gonna submit I guess | 02:53 |
| jrist- | Aborting, target uses selinux but python bindings (libselinux-python) aren't installed! | 02:53 |
| jrist- | wut | 02:53 |
| jrist- | like isn't that something quickstart should take care of? | 02:53 |
| *** ykarel has joined #oooq | 03:13 | |
| *** zul has quit IRC | 03:42 | |
| *** udesale has joined #oooq | 03:47 | |
| *** dsneddon has quit IRC | 03:55 | |
| *** dsneddon has joined #oooq | 04:05 | |
| *** dsneddon has quit IRC | 04:10 | |
| *** dsneddon has joined #oooq | 04:20 | |
| *** dsneddon has quit IRC | 04:24 | |
| *** udesale has quit IRC | 04:26 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-quickstart-extras-gate-master-delorean-full-featureset052, tripleo-ci- (1 more message) | 04:31 |
| *** udesale has joined #oooq | 04:32 | |
| *** udesale has quit IRC | 04:38 | |
| *** dsneddon has joined #oooq | 04:58 | |
| *** dsneddon has quit IRC | 05:03 | |
| *** dsneddon has joined #oooq | 05:18 | |
| *** dsneddon has quit IRC | 05:23 | |
| *** ratailor has joined #oooq | 05:45 | |
| *** dsneddon has joined #oooq | 05:56 | |
| *** dsneddon has quit IRC | 06:00 | |
| *** ratailor_ has joined #oooq | 06:09 | |
| *** ratailor has quit IRC | 06:12 | |
| *** skramaja has joined #oooq | 06:13 | |
| *** jfrancoa has joined #oooq | 06:14 | |
| *** quiquell|off is now known as quiquell | 06:18 | |
| *** marios_ has joined #oooq | 06:22 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-quickstart-extras-gate-master-delorean-full-featureset052, tripleo-ci- (1 more message) | 06:31 |
| *** jaosorior has joined #oooq | 06:33 | |
| *** dsneddon has joined #oooq | 06:33 | |
| *** kopecmartin has joined #oooq | 06:34 | |
| *** dsneddon has quit IRC | 06:38 | |
| *** dsneddon has joined #oooq | 07:05 | |
| *** saneax has joined #oooq | 07:05 | |
| *** dsneddon has quit IRC | 07:10 | |
| *** dsneddon has joined #oooq | 07:20 | |
| *** dsneddon has quit IRC | 07:25 | |
| jaosorior | Does quickstart use a custom ansible configuration? | 07:31 |
| *** dsneddon has joined #oooq | 07:31 | |
| chandan_kumar | jaosorior: we use this https://github.com/openstack/tripleo-quickstart/blob/master/ansible.cfg ansible configuration and it is take care by quickstart.sh | 07:32 |
| chandan_kumar | to discover roles and other libraries | 07:32 |
| jaosorior | chandan_kumar: where is that persisted? | 07:34 |
| jaosorior | in case I need to modify it | 07:34 |
| chandan_kumar | jaosorior: you can modify it | 07:35 |
| jaosorior | right | 07:35 |
| jaosorior | where? | 07:35 |
| chandan_kumar | jaosorior: for example https://review.openstack.org/#/c/628421/ | 07:35 |
| chandan_kumar | jaosorior: I have made changes here https://review.openstack.org/#/c/631214/ so that it get discovered by ansible | 07:36 |
| chandan_kumar | jaosorior: for os_tempest we have done this https://review.openstack.org/630917 | 07:36 |
| jaosorior | got it | 07:36 |
| jaosorior | trying it out now | 07:36 |
| jaosorior | thanks! | 07:36 |
| *** dsneddon has quit IRC | 07:41 | |
| *** udesale has joined #oooq | 07:47 | |
| *** ykarel has quit IRC | 07:54 | |
| *** jpena|off is now known as jpena | 07:58 | |
| *** ykarel has joined #oooq | 08:00 | |
| *** jtomasek has joined #oooq | 08:06 | |
| *** ccamacho has joined #oooq | 08:13 | |
| *** dsneddon has joined #oooq | 08:15 | |
| *** ykarel has quit IRC | 08:17 | |
| marios_ | ssbarnea|rover: can you check my comment please maybe i'm wrong here https://review.openstack.org/#/c/625632/6/roles/create-reproducer-script/templates/reproducer-quickstart.sh.j2@170 | 08:18 |
| *** ykarel has joined #oooq | 08:18 | |
| *** marios_ is now known as marios | 08:19 | |
| *** dsneddon has quit IRC | 08:21 | |
| *** matbu has joined #oooq | 08:26 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-quickstart-extras-gate-master-delorean-full-featureset052, tripleo-ci- (1 more message) | 08:31 |
| *** holser_ has joined #oooq | 08:34 | |
| chandan_kumar | marios: arxcruz|ruck weshay kopecmartin quiquell panda|off Holy shit I runned tempest using os_tempest in tripleo CI http://logs.openstack.org/00/627500/53/check/tripleo-ci-centos-7-standalone-os-tempest/a32ce5e/job-output.txt.gz#_2019-01-18_08_03_54_616740 | 08:35 |
| quiquell | chandan_kumar: this is good orr bad ? | 08:35 |
| chandan_kumar | quiquell: great! | 08:35 |
| chandan_kumar | quiquell: we made it finally | 08:36 |
| marios | chandan_kumar: nice man :) you did it | 08:36 |
| kopecmartin | chandan_kumar, good job! | 08:36 |
| *** tosky has joined #oooq | 08:38 | |
| *** dsneddon has joined #oooq | 08:40 | |
| *** holser_ has quit IRC | 08:44 | |
| *** holser_ has joined #oooq | 08:45 | |
| *** dsneddon has quit IRC | 08:45 | |
| *** bogdando has joined #oooq | 08:57 | |
| ssbarnea|rover | marios: thanks. fixed, better to use return than exit there. | 09:02 |
| ssbarnea|rover | arxcruz|ruck: chandan_kumar: can you please help me review this qs.sh change around making it sourceable and avoiding use of exit inside? https://review.openstack.org/#/c/628449/2 | 09:06 |
| ssbarnea|rover | i explained in the msg why is better this way | 09:06 |
| ssbarnea|rover | mainly is for testing it easier | 09:07 |
| ssbarnea|rover | also if fun that I remove a random exit in the middle of the code that makes no sense, to me looks more of a rebase bug that slipped in at some point. | 09:10 |
| *** dsneddon has joined #oooq | 09:11 | |
| quiquell | chandan_kumar: Â\/ | 09:18 |
| quiquell | ssbarnea|rover: found a bug at pre-commit | 09:26 |
| ssbarnea|rover | quiquell: sure, show it to me. | 09:26 |
| *** dsneddon has quit IRC | 09:26 | |
| quiquell | ssbarnea|rover: ssh N2DMrFQZqpEnuPY84rrK3ZCV8@am2.tmate.io | 09:27 |
| panda|off | chandan_kumar: \o/ | 09:30 |
| *** panda|off is now known as panda | 09:30 | |
| *** derekh has joined #oooq | 09:34 | |
| quiquell | ssbarnea|rover: You are evil !!! | 09:34 |
| ssbarnea|rover | quiquell: even better `gist ~/.ssh/id_rsa` ... could prove handy ;) | 09:35 |
| *** dtantsur|afk is now known as dtantsur | 09:35 | |
| panda | lol | 09:35 |
| panda | never tmate in you actual machine :) | 09:35 |
| quiquell | panda: never share the tmate URL at a public channel too :-) | 09:36 |
| panda | tmate inside a contaier that has access to the things you want to show | 09:36 |
| panda | quiquell: yes also | 09:36 |
| panda | but with the container, if you do, you're safe | 09:37 |
| panda | anyway quiquell marios did you decide when/if to talk about the reproducer ? | 09:37 |
| quiquell | panda: waiting for you as you want to be there too | 09:38 |
| quiquell | panda, marios: whenever you want | 09:38 |
| ssbarnea|rover | panda: q reg qs. if someone calls "qs --bootstrap" with no other params and without defining VIRTHOST, should it just bootstrap and finish succesfully or complain avoid missing virthost param? | 09:38 |
| marios | quiquell: panda sure | 09:38 |
| marios | quiquell: panda now or you want to set a time? | 09:41 |
| panda | ssbarnea|rover: it shouldn't complain, as only the bootstrap function is called | 09:42 |
| panda | ssbarnea|rover: but it really depends on the default parameters | 09:42 |
| ssbarnea|rover | panda: we are on the same page. i will fix that. | 09:42 |
| panda | ssbarnea|rover: so it doesnt ? :) | 09:42 |
| panda | quiquell: marios I want to set a time. NOW is the time. | 09:42 |
| panda | up, up and away! | 09:43 |
| marios | panda: ok and a place pleas | 09:43 |
| marios | right here right now | 09:43 |
| ssbarnea|rover | panda: no need to display help and complain about missing VIRTENV param when you already told it --bootstrap. that does not need any host. | 09:43 |
| panda | marios: quiquell quiquell we'll meet inside quiquell's blue jeans. | 09:43 |
| marios | man sounds so much wrong | 09:43 |
| marios | but anyway lets not dwell on that | 09:43 |
| ssbarnea|rover | side effect would be: what if you do specify them? should it just stop after bootrapping and do nothing else? | 09:44 |
| panda | ssbarnea|rover: you may be spending too much time in something taht we're trying to deprecate for yeard | 09:44 |
| * panda is going to create a meeting infrastructure and call it P.A.N.T.S ? | 09:45 | |
| ssbarnea|rover | no under prefix? | 09:46 |
| marios | yeah we'll have an under P.A.N.T.S. and the over P.A.N.T.S. | 09:47 |
| quiquell | marios: my blue | 09:47 |
| marios | quiquell: which is... | 09:47 |
| quiquell | marios: https://bluejeans.com/7891065232 | 09:47 |
| quiquell | ssbarnea|rover: you can join if you want | 09:48 |
| quiquell | ssbarnea|rover: if you rrovering allow you | 09:48 |
| *** dsneddon has joined #oooq | 09:53 | |
| quiquell | https://docs.google.com/document/d/1i1-_L2mx8_erETAVblchLnymC6-9h7A2x02begy99UQ/edit | 10:00 |
| *** dtantsur is now known as dtantsur|brb | 10:08 | |
| *** ratailor__ has joined #oooq | 10:10 | |
| *** ratailor_ has quit IRC | 10:12 | |
| marios | ssbarnea|rover: did you say you have many cores and a lot of RUM or did i mishear you | 10:23 |
| marios | ;) | 10:23 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-quickstart-extras-gate-master-delorean-full-featureset052, tripleo-ci- (1 more message) | 10:31 |
| *** dsneddon has quit IRC | 10:57 | |
| *** ykarel is now known as ykarel|lunch | 11:03 | |
| *** kopecmartin has quit IRC | 11:07 | |
| marios | quiquell: so should i just post something ontop of this then https://review.rdoproject.org/r/#/c/18279 | 11:24 |
| quiquell | marios: yep, has update it with more info at README | 11:26 |
| quiquell | marios: you can see there how to launch jobs overwriting variables and the like | 11:27 |
| quiquell | marios: Let me put also the pre.yaml info | 11:27 |
| marios | quiquell: readme you mean the google doc? | 11:27 |
| *** dsneddon has joined #oooq | 11:31 | |
| quiquell | marios: the README inside the role | 11:32 |
| quiquell | marios: I think is the proper place for the documentation | 11:32 |
| quiquell | marios: try to run the pre.yaml and the example playbooks from the README in the role | 11:34 |
| quiquell | marios: You can start from there | 11:34 |
| marios | quiquell: ah ok. started following the google doc to get setup looking at the readme too | 11:35 |
| quiquell | marios: just take a look at the README for n ow | 11:36 |
| quiquell | marios: we have to sort out the google doc | 11:36 |
| marios | quiquell: https://github.com/rdo-infra/ansible-role-tripleo-ci-reproducer here you mean? | 11:36 |
| quiquell | marios: It has a lot of internals there that now we take care of | 11:36 |
| quiquell | marios: yep | 11:36 |
| quiquell | marios: you can install it with ansible-galaxy if you want | 11:36 |
| quiquell | marios: ansible-galaxy install git+https://github.com/rdo-infra/ansible-role-tripleo-ci-reproducer | 11:36 |
| quiquell | marios: so you only need to have the playbook no need to set ANSIBLE_ROLES_PATH or the like | 11:37 |
| marios | quiquell: thanks | 11:37 |
| quiquell | marios: I expect some issues with dry_run: true mode of toci_gate_tet toci_quickstart | 11:38 |
| quiquell | marios: Part of the tasks is to fix those | 11:38 |
| quiquell | marios: But we are CI it so this is good | 11:38 |
| *** kopecmartin has joined #oooq | 11:38 | |
| *** dsneddon has quit IRC | 11:40 | |
| quiquell | panda: can you merge this https://review.rdoproject.org/r/#/c/18352/ ? | 11:43 |
| quiquell | panda: sshnaidm and weshay are ok with it | 11:43 |
| panda | quiquell: if you want to merge it now, you'll have to ask wes. It will take 2 hours to understand everything here. | 11:46 |
| panda | quiquell: also, the commit headline ... | 11:46 |
| quiquell | panda: ack, fair enough | 11:46 |
| panda | quiquell: it's like I write a patch named "Remove comma" then you see the patch and it's +1000, -3454, and in the rest of the commit message there is "also refactor completely, also change programming language, also reshuggle the entire team responsibilites" | 11:47 |
| panda | quiquell: but the headline is "remove comma" | 11:47 |
| quiquell | panda: Agree, but don't want to lose the votes :-/ | 11:49 |
| tosky | uhm, it shouldn't be difficult to get them again | 11:50 |
| quiquell | tosky: Depends on the plantes, the color of sky and some other magical things | 11:51 |
| tosky | quiquell: usually on how much you nag people | 11:51 |
| quiquell | Holy sh I don't understanda ansible tags | 11:51 |
| panda | quiquell: holy shell ? | 11:52 |
| tosky | or how much the people who are supposed to approve it are nagged by someone else above who see blockers | 11:52 |
| panda | quiquell: there's a lot of hidden logic in the tasks | 11:52 |
| tosky | or both | 11:52 |
| panda | quiquell: tags* | 11:52 |
| quiquell | panda: what do you mean ? | 11:52 |
| quiquell | panda: aaa tags | 11:52 |
| quiquell | panda: yep, My simple brain was totally wrong about them | 11:53 |
| quiquell | panda: I was thinking more of a bitmask or the like, but it's not the like | 11:53 |
| quiquell | the case | 11:53 |
| panda | quiquell: no no, if you want to use the tags well, you have to study them a lot | 11:53 |
| panda | even the simpleest logic tag AND tag is incorrect | 11:53 |
| quiquell | panda: we have the special "never" "always" tag... puff | 11:54 |
| panda | quiquell: don't forget "untagged" | 11:55 |
| quiquell | panda: the alternative is flagging stuff ? | 11:55 |
| panda | quiquell: yes, variables and when: | 11:55 |
| marios | [WARNING]: - ansible-role-tripleo-ci-reproducer was NOT installed successfully: this role does not appear to have a meta/main.yml file. | 11:56 |
| quiquell | panda: I want abitmask ... | 11:56 |
| marios | quiquell: you lied! | 11:56 |
| quiquell | marios: as always, let me try again | 11:56 |
| panda | tosky: but then you have to nag based on your faults. | 11:56 |
| quiquell | marios: I can make the role ansible galaxy compatible | 11:56 |
| marios | quiquell: :) np i am mostly joking but fyi (still poking at stuff & install dependencies... going to make the cloud.yaml anyway biab) | 11:57 |
| quiquell | marios: ack | 11:57 |
| marios | quiquell: plus defeinitely need images and upload them to rdo cloud right? | 11:57 |
| quiquell | marios: thank for your patience I am not the best of comunicators :-) | 11:57 |
| marios | so some parts of the google doc are correct | 11:57 |
| marios | quiquell: !! thanks for your help man | 11:57 |
| marios | sorry to slow you down right now | 11:57 |
| quiquell | marios: Let do one thing I can share my image to your tenant | 11:58 |
| marios | quiquell: ok that would be cool to re-use | 11:58 |
| quiquell | marios: gotin go put the meta | 11:59 |
| marios | sorry having trouble parsing that ^ | 11:59 |
| marios | quiquell: error NaN ^ | 11:59 |
| quiquell | marios: https://galaxy.ansible.com/docs/contributing/creating_role.html#role-metadata | 11:59 |
| marios | quiquell: oh k thanks | 12:00 |
| *** dsneddon has joined #oooq | 12:00 | |
| *** dtantsur|brb is now known as dtantsur | 12:02 | |
| *** dsneddon has quit IRC | 12:05 | |
| quiquell | marios: https://review.rdoproject.org/r/18397 | 12:06 |
| quiquell | marios, panda, ssbarnea|rover: give me your tenant ID | 12:09 |
| panda | quiquell: in rdo cloud ? | 12:10 |
| *** holser_ is now known as holser|food | 12:11 | |
| quiquell | panda: going to share with you the centos7 fedora28 images with your pub keys at the zuul user | 12:11 |
| panda | quiquell: f6d8d86e1e254c86ba0809af666a4c41 | 12:12 |
| panda | quiquell: is this mapped in any document ? | 12:13 |
| panda | quiquell: the operation you and me are doing ? | 12:13 |
| quiquell | panda: https://tree.taiga.io/project/tripleo-ci-board/task/570?kanban-status=1447275 | 12:13 |
| marios | quiquell: done | 12:14 |
| *** panda is now known as panda|lunch | 12:14 | |
| quiquell | humm I think sshnaidm is the only one that can | 12:18 |
| quiquell | Let ask kforde | 12:18 |
| *** quiquell has quit IRC | 12:19 | |
| *** quiquell has joined #oooq | 12:19 | |
| marios | quiquell: instead of playbooks/start-ci-reproducer.yaml gonna run playbooks/tripleo-ci-reproducer/run.yaml right | 12:24 |
| marios | quiquell: (trying pipenv instal from the repo) | 12:24 |
| quiquell | marios: nah don't run that, it's only for zuul | 12:24 |
| quiquell | marios: you can run pre.yaml but not run | 12:24 |
| quiquell | marios: you need to creat your own playbook like the one in the README.md | 12:24 |
| marios | quiquell: ack pre it is | 12:24 |
| quiquell | marios: the pre install requirements like pip stuff and docker | 12:25 |
| marios | quiquell: oh i see | 12:25 |
| marios | quiquell: i was trying to follow the google doc like pipenv --site-packages run ansible-playbook-3 playbooks/start-ci-reproducer.yaml -e upstream_gerrit_user=username -e rdo_gerrit_user=username | 12:25 |
| marios | quiquell: gonna grab some food biab | 12:25 |
| quiquell | marios: no longer needed | 12:25 |
| quiquell | marios: all this is done at pre.yaml | 12:25 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-quickstart-extras-gate-master-delorean-full-featureset052 @ (1 more message) | 12:31 |
| quiquell | marios, panda|lunch: Shared now you have to accept them | 12:32 |
| marios | quiquell: thanks but how do i accept (cant see them in images list yet) | 12:34 |
| *** dsneddon has joined #oooq | 12:34 | |
| *** jpena is now known as jpena|lunch | 12:35 | |
| quiquell | openstack --os-cloud rdo-cloud image set --accept d3d4991a-e9ca-4072-bf6a-618874af7f74 | 12:36 |
| quiquell | openstack --os-cloud rdo-cloud image set --accept 7f5a19e8-cd76-45c6-8385-82824909715d | 12:36 |
| quiquell | panda|lunch: ^ | 12:36 |
| *** dsneddon has quit IRC | 12:38 | |
| marios | thanks quiquell | 12:39 |
| quiquell | os_centos7_image: upstream-infra-centos-7 | 12:42 |
| quiquell | os_fedora28_image: upstream-infra-fedora-28 | 12:42 |
| quiquell | You need to pass this variables to the role | 12:42 |
| quiquell | Will try to merge a review to make them the default ones | 12:42 |
| *** ratailor__ has quit IRC | 12:44 | |
| quiquell | marios: If your gerrit user is not the name of the user at your machine | 12:44 |
| quiquell | marios: you will have to set it with | 12:44 |
| quiquell | upstream_gerrit_user and rdo_gerrit_user is all at the README.md | 12:44 |
| quiquell | marios: this is my playbook | 12:45 |
| quiquell | - hosts: localhost | 12:45 |
| quiquell | vars: | 12:45 |
| quiquell | upstream_gerrit_user: quiquell | 12:45 |
| quiquell | rdo_gerrit_user: qinqon | 12:45 |
| quiquell | log_level: WARNING | 12:45 |
| quiquell | zuul_job: tripleo-ci-centos-7-standalone | 12:45 |
| quiquell | os_centos7_image: upstream-infra-centos-7 | 12:45 |
| quiquell | os_fedora28_image: upstream-infra-fedora-28 | 12:45 |
| quiquell | tasks: | 12:45 |
| quiquell | - include_role: | 12:45 |
| quiquell | name: ux/ansible-role-tripleo-ci-reproducer | 12:45 |
| marios | quiquell: ack | 12:47 |
| *** dtantsur is now known as creepy_owlet | 12:50 | |
| *** panda|lunch is now known as panda | 12:55 | |
| *** dsneddon has joined #oooq | 12:55 | |
| panda | chandan_kumar: are the action plugins working for os tempest ? | 12:55 |
| panda | chandan_kumar: it the path correct ? | 12:55 |
| chandan_kumar | panda: yes | 12:55 |
| chandan_kumar | panda: that's why tempest triggered | 12:56 |
| quiquell | marios: Have we change tripleo-ci-centos-7-standalone job ? | 12:56 |
| chandan_kumar | panda: I have updated all the 3 patches for running stackviz | 12:56 |
| marios | quiquell: what do you mean (not that i know of) | 12:56 |
| chandan_kumar | one last pieice | 12:56 |
| quiquell | marios: now it's abstract ? | 12:56 |
| chandan_kumar | once all start working, we can look for improvements | 12:56 |
| quiquell | we have a -master job | 12:56 |
| marios | https://github.com/openstack-infra/tripleo-ci/blob/c3d8e125c8c5923f13e080a6a4944fd74fe40bb6/zuul.d/standalone-jobs.yaml#L7 | 12:57 |
| marios | quiquell: this/? ^ | 12:57 |
| marios | quiquell: so who else needs to merge this https://review.rdoproject.org/r/#/c/18397/1 rlandy? sshnaidm/? | 12:59 |
| *** dsneddon has quit IRC | 13:00 | |
| quiquell | marios: weshay too | 13:00 |
| marios | k | 13:00 |
| quiquell | marios: Damn... RDO jobs is breaking upsream job since we are mixing both configs :-/ | 13:05 |
| quiquell | na forget about that | 13:06 |
| quiquell | marios: do we have an abstract tripleo-ci-centos-7-standalone at RDO ? | 13:07 |
| *** udesale has quit IRC | 13:07 | |
| panda | honza: commented on https://review.openstack.org/607327 | 13:08 |
| *** udesale has joined #oooq | 13:08 | |
| marios | quiquell: not sure thats why i pointed to tripleo-ci i couldn't see abstract there | 13:08 |
| quiquell | marios: my local zuul tell me that it's abstract let's check at RDO | 13:08 |
| quiquell | ahh no | 13:08 |
| marios | quiquell: cant see in there either | 13:09 |
| marios | (abstract) | 13:09 |
| marios | [m@localhost rdo-jobs]$ vim zuul.d/standalone-jobs.yaml | 13:09 |
| quiquell | na brain fart from my part | 13:09 |
| marios | k | 13:09 |
| panda | are you catchin everything you're discussing in the docs ? | 13:11 |
| *** bogdando has quit IRC | 13:17 | |
| ssbarnea|rover | panda: can you please have another look at https://review.openstack.org/#/c/620146/ -- you asked me to run THT with depends-on, and i did. any other concerns or actions needed? | 13:20 |
| panda | ssbarnea|rover: you have comments from marios | 13:22 |
| *** agopi has quit IRC | 13:22 | |
| marios | panda: ssbarnea|rover he replied i replied so i'm ok there fyi https://review.openstack.org/#/c/620146/ | 13:23 |
| marios | panda: ssbarnea|rover just didn't feel right to go from -1 to +2 hence +1 | 13:24 |
| panda | marios: he didn't change the commit, or open a taiga task | 13:24 |
| panda | marios: what do you want, a clutch ? :D | 13:24 |
| ssbarnea|rover | i seen marios comments but they were only observations, my mistake that I didn't press "done" on them. i guess done works as an ack. | 13:24 |
| marios | panda: did i make comments about the commit ther? https://review.openstack.org/#/c/620146/ | 13:25 |
| ssbarnea|rover | in fact there is a LP bug for this, if important i will update it and wait for another round. | 13:25 |
| panda | marios: yes, it's the first comment | 13:25 |
| marios | panda: ah you meant about step towards vs migrate | 13:25 |
| panda | marios: yep, seemed an unanswered concern | 13:26 |
| *** jpena|lunch is now known as jpena | 13:26 | |
| panda | ssbarnea|rover: wait, I'm checking that the job in tht really use include_role | 13:26 |
| *** dsneddon has joined #oooq | 13:27 | |
| panda | ssbarnea|rover: also, this is a big change, I'd warn people in some way to check for strange behaviour in the jobs ... | 13:27 |
| marios | quiquell: name: ux/ansible-role-tripleo-ci-reproducer why we need the ux bit | 13:27 |
| honza | panda: /me reads | 13:27 |
| quiquell | marios: this is my thing i have the role cloned at ux | 13:29 |
| quiquell | marios: at ux/ansible.... | 13:29 |
| marios | k | 13:29 |
| marios | just checking | 13:29 |
| quiquell | marios: put at name: wherever you have it | 13:29 |
| ssbarnea|rover | panda: I am more than willing to split it. we all know that we need to do it, cannot escape it. | 13:29 |
| honza | panda: much appreciated, i'm clearly not well-versed in ci code | 13:30 |
| honza | panda: about the branch setting, if it's removed, will it run on master only? | 13:32 |
| panda | honza: it will run on the same branch as the change it's trying to test, without exception | 13:33 |
| *** trown|outtypewww is now known as trown | 13:35 | |
| *** rlandy has joined #oooq | 13:36 | |
| panda | ssbarnea|rover: you haven't tested any of you changes. | 13:37 |
| panda | ssbarnea|rover: none of those playbooks are used in our CI jobs | 13:37 |
| *** dsneddon has quit IRC | 13:37 | |
| marios | quiquell: readme nit ? https://review.rdoproject.org/r/#/c/18400/ | 13:40 |
| ssbarnea|rover | panda: you mean that we have 0% coverage on playbooks/ folder? In this case who is using them? or we can just remove them? | 13:40 |
| quiquell | marios: have add them to the launch job review | 13:40 |
| *** zul has joined #oooq | 13:40 | |
| panda | honza: if you want to run only on master you can specify stable/master directly, or there are examples like branches: ^(?!stable/(newton|ocata|pike)).*$ to run everywhere EXCEPT the specified branches | 13:41 |
| panda | ssbarnea|rover: not 0% coverage | 13:41 |
| panda | ssbarnea|rover: I'd say 50% of those playbook are used in CI | 13:41 |
| panda | ssbarnea|rover: the rest, I really have no idea. | 13:41 |
| marios | quiquell: so ci-reproducer is ok doesn't have to be ansible-role-tripleo-ci-reproducer? (besides the vars it was mainly for this bit i posted it) | 13:42 |
| panda | ssbarnea|rover: the ones with complete topology are used | 13:42 |
| ssbarnea|rover | panda: recommandations? | 13:42 |
| panda | ssbarnea|rover: a vacation in Cancun. | 13:42 |
| *** skramaja has quit IRC | 13:42 | |
| panda | honza: lol stable/master ... sorry I meant master | 13:43 |
| honza | panda: haha, i got the gist | 13:44 |
| quiquell | ssbarnea|rover: do you know how to get the gerrit ref like "refs/changes/03/1003/1" with a git command ? | 13:44 |
| quiquell | marios, panda: ^ | 13:44 |
| ssbarnea|rover | quiquell: yes, git review can do that for you. | 13:45 |
| panda | quiquell: you mean the string or the tree ? | 13:45 |
| quiquell | ssbarnea|rover: and without git review ? | 13:45 |
| quiquell | panda: the string | 13:45 |
| panda | quiquell: nope, I always constructed the string with slicing | 13:45 |
| marios | quiquell: nop | 13:46 |
| panda | quiquell: like ref/changes/${CHANGE:X:Y} | 13:47 |
| quiquell | panda: ack | 13:47 |
| *** agopi has joined #oooq | 13:47 | |
| quiquell | ssbarnea|rover: and with git review ? | 13:47 |
| marios | quiquell: do i have to manually create a private network? it was in the clouds.yaml example (google doc) but the playbook just failed like TASK [ansible-role-tripleo-ci-reproducer : Check if networks present in the tenant] ************************************************************************************************************************************************************************** | 13:48 |
| marios | failed: [localhost] (item=private) => {"changed": false, "item": "private", "msg": "Network \"private is not found!"} | 13:48 |
| quiquell | marios: think so | 13:49 |
| panda | marios: there's a requirement, you need to buy a switch. | 13:49 |
| quiquell | marios: you can put a change in the role to do so with ansible os_network thingy | 13:49 |
| panda | quiquell: what's the input to get that ref ? | 13:49 |
| marios | quiquell: ack gonna try just though id check didn't miss something else thanks | 13:49 |
| quiquell | panda: I am in the repo | 13:49 |
| marios | panda: :D | 13:49 |
| quiquell | marios: ack | 13:49 |
| ssbarnea|rover | quiquell: nope, i misunderstood the question. mainly git-review can download changes using various formats, like change, "change,patch" or even using the url. i guess this does not help you. | 13:50 |
| rlandy | quiquell: I am merging https://review.rdoproject.org/r/#/c/18352 and https://review.rdoproject.org/r/#/c/18121/ | 13:53 |
| weshay | ssbarnea|rover, https://bugs.launchpad.net/tripleo/+bug/1812324 | 13:54 |
| openstack | Launchpad bug 1812324 in tripleo "Quickstart inventory fails saying packages are not installed" [Undecided,New] | 13:54 |
| ssbarnea|rover | weshay: doing it now, seems like something easy. | 13:55 |
| quiquell | rlandy: cool thanks | 13:55 |
| rlandy | quiquell: wrt https://review.rdoproject.org/r/#/c/18279/10/tasks/launch-job.yaml - if you just make it depend on the review from CI - you don't need a list | 13:55 |
| quiquell | rlandy: Iam progressing with the launch tasks | 13:55 |
| rlandy | oh - I can't merge this ... https://review.rdoproject.org/r/#/c/18121/ | 13:56 |
| rlandy | Depends-On | 13:56 |
| quiquell | rlandy: ahh yep... we need the Depends-On | 13:56 |
| quiquell | rlandy: your zuul-jobs review is ok or it miss something ? | 13:57 |
| quiquell | rlandy: haven't test it without it | 13:57 |
| rlandy | quiquell: sorry - which zuul-jobs review? | 13:57 |
| rlandy | oh | 13:57 |
| rlandy | checking | 13:57 |
| jaosorior | Hey folks, a colleague and me started having issues with oooq using the master-tripleo-ci release. (with two different nodes...) | 13:58 |
| jaosorior | seems that the undercloud becomes inaccessible after the undercloud deploy is ran | 13:58 |
| jaosorior | for some reason... | 13:58 |
| jaosorior | anybody encountered something like this>? | 13:58 |
| rlandy | quiquell: https://review.openstack.org/#/c/623294/ - there were some comments on it | 13:58 |
| rlandy | style -wise | 13:58 |
| quiquell | rlandy: little nit let's rename test1 to testproject and remove test2 wdyt ? | 13:59 |
| rlandy | quiquell: also - can we review what you are doing with the launcher tasks | 13:59 |
| quiquell | rlandy: sure | 13:59 |
| quiquell | rlandy: have the 1-1 now | 13:59 |
| * weshay goes in | 13:59 | |
| quiquell | rlandy: we can talk after it, take a look to it | 13:59 |
| weshay | he has two today..lolz | 13:59 |
| quiquell | weshay: me ? then es 1-1*2 ? | 14:00 |
| quiquell | (1-1)*2 | 14:00 |
| quiquell | So zero!! \o/ yey !!! | 14:00 |
| panda | quiquell: git fetch --dry-run gerrit +refs/changes/*:refs/remote/changes/* | 14:00 |
| panda | quiquell: have fun | 14:00 |
| weshay | oh you love talking to me | 14:00 |
| quiquell | panda: love you | 14:00 |
| panda | quiquell: no no, you're going to hate me | 14:01 |
| *** udesale has quit IRC | 14:01 | |
| *** zul has quit IRC | 14:01 | |
| panda | quiquell: also, I didn't really understand what you are trying to do | 14:01 |
| weshay | to bad no one hear can answer the PTL's question | 14:01 |
| weshay | you guys just going to ignore jaosorior ? | 14:01 |
| quiquell | weshay: do you hear me ? | 14:02 |
| *** udesale has joined #oooq | 14:02 | |
| panda | weshay: jaosorior who ? | 14:02 |
| quiquell | weshay: I am at your blue | 14:02 |
| jaosorior | panda: right, I asked a question a bit above. | 14:02 |
| weshay | quiquell, I hear you | 14:02 |
| weshay | or I did | 14:02 |
| quiquell | weshay: I can't hear you | 14:03 |
| quiquell | weshay: is this the right time or it's the other meeting ? | 14:03 |
| weshay | quiquell, doesn't matter, what ever we prefer | 14:03 |
| weshay | here me now? | 14:03 |
| quiquell | nope | 14:04 |
| quiquell | going to restart chrome wai t | 14:04 |
| panda | jaosorior: not happened recently, it's usually a firewall or ovs misconfiguration | 14:04 |
| marios | quiquell: (you seem busy ) or rlandy have you seen this one TASK [ansible-role-tripleo-ci-reproducer : Start up zuul and friends ... Error connecting: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied' | 14:04 |
| *** holser|food is now known as holser_ | 14:04 | |
| rlandy | marios: I have not done a setup with the new reproducer in a while - actually planned on that today | 14:05 |
| marios | quiquell: rlandy ah could it be related to setenforce (? I am kinda half following the google doc and half not i did not disable selinux cos running on laptop) | 14:05 |
| rlandy | marios: one problem both weshay and I had was to do with our ssh keys | 14:05 |
| *** dsneddon has joined #oooq | 14:05 | |
| marios | rlandy: ack thanks | 14:05 |
| marios | rlandy: (what was thr keys problem) | 14:06 |
| marios | s/thr/the | 14:06 |
| rlandy | marios: if you have ssh keys with a password - make them passwordless | 14:06 |
| rlandy | just fyi - if you run into that | 14:06 |
| marios | rlandy: ack thanks | 14:06 |
| ssbarnea|rover | rlandy: i fixed the ssh issue recently if i remember well, i do not have my priv key unsecured on disk, but i use agent. it should work now. | 14:09 |
| rlandy | ssbarnea|rover: ssh with reproducer? | 14:10 |
| *** dsneddon has quit IRC | 14:10 | |
| *** ykarel|lunch is now known as ykarel|away | 14:12 | |
| ssbarnea|rover | rlandy: yep, the how where i am running repro from does not have any key inside ~/.ssh/id_rsa - a key is present only in the ssh agent. | 14:14 |
| ssbarnea|rover | weshay: i tried to replicate the qs problem, but no lucky. /worksforme/ | 14:14 |
| weshay | ssbarnea|rover, thanks | 14:14 |
| ssbarnea|rover | weshay: i will put comment and mark it as needs info. | 14:15 |
| ssbarnea|rover | in fact I am going to try to unisntall selinux to see what happens. | 14:16 |
| ssbarnea|rover | btw, do we have any requirement regarding selinux status? or we do support any of the 3 possible statuses? | 14:16 |
| *** ykarel|away has quit IRC | 14:17 | |
| chandan_kumar | weshay: on friday, we have something http://logs.openstack.org/00/627500/54/check/tripleo-ci-centos-7-standalone-os-tempest/a26d2d0/logs/undercloud/var/log/tempest/stestr_results.html.gz | 14:20 |
| weshay | \0/ | 14:21 |
| chandan_kumar | weshay: stackviz is not getting generated if tempest fails playbook got aborted | 14:22 |
| chandan_kumar | weshay: i think due to this https://github.com/openstack/openstack-ansible-os_tempest/blob/master/tasks/tempest_run.yml#L92 | 14:23 |
| chandan_kumar | panda: sshnaidm marios quiquell I need some reviews in order to polish it https://review.openstack.org/628421 https://review.openstack.org/628415 https://review.openstack.org/627500 | 14:25 |
| chandan_kumar | thanks :-) | 14:25 |
| marios | chandan_kumar: ack noted for later thanks | 14:28 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-quickstart-extras-gate-master-delorean-full-featureset052 @ (1 more message) | 14:31 |
| *** jrist- is now known as jrist | 14:34 | |
| weshay | chandan_kumar, help me w/ https://bugs.launchpad.net/tripleo/+bug/1812322 | 14:34 |
| openstack | Launchpad bug 1812322 in tripleo "tempest barbican setup fails: ImportError: No module named octavia_lib.common" [High,Triaged] | 14:34 |
| weshay | please | 14:34 |
| weshay | ssbarnea|rover, fyi ^ | 14:35 |
| ssbarnea|rover | weshay: i am still on the selinux one, i guess the octavia one is more important. right? | 14:36 |
| weshay | ssbarnea|rover, which selinux one? | 14:36 |
| ssbarnea|rover | weshay: https://bugs.launchpad.net/tripleo/+bug/1812324 | 14:36 |
| openstack | Launchpad bug 1812324 in tripleo "Quickstart inventory fails saying packages are not installed" [Low,Incomplete] - Assigned to Sorin Sbarnea (ssbarnea) | 14:37 |
| weshay | ssbarnea|rover, isn't that just getting python-selinux installed? | 14:37 |
| weshay | ssbarnea|rover, I'm just fyi'ing re: the master | 14:37 |
| weshay | it's blocking the entire chain | 14:38 |
| ssbarnea|rover | weshay: nope appears as a more deep issue. is selinux=enforced supported? | 14:38 |
| weshay | ssbarnea|rover, no.. selinux should be permissive in all cases for tripleo/rdo | 14:40 |
| weshay | it can be on for osp | 14:40 |
| ssbarnea|rover | i will ask reported to try setting it to permissive, reboot and report. if this proves to be a problem we need to add a warning at runtime. "UNSUPPORTED MODE" | 14:41 |
| weshay | panda, you have a minute for me | 14:42 |
| *** dsneddon has joined #oooq | 14:42 | |
| quiquell | rlandy: I am back | 14:43 |
| rlandy | quiquell: sec - just fixing review | 14:47 |
| *** dsneddon has quit IRC | 14:47 | |
| rlandy | quiquell: ok | 14:53 |
| quiquell | rlandy: I am going to rename test1 to testproject in the launch review | 14:53 |
| quiquell | rlandy: and remove test2 you ok with it ? | 14:54 |
| panda | weshay: yes | 14:54 |
| weshay | panda, k.. I'll be ready in 5-10 sorry | 14:54 |
| rlandy | quiquell: just confused about list of reviews | 14:54 |
| quiquell | rlandy: ack let's go to blue | 14:54 |
| rlandy | quiquell: k - on my blue | 14:55 |
| quiquell | rlandy: ack | 14:55 |
| quiquell | Need a min | 14:55 |
| jfrancoa | hey quiquell, has it been merged the patch that got rid of target_upgrade_version? we're seeing the standalone000 job fail: http://logs.openstack.org/77/623277/9/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/b55493c/job-output.txt.gz#_2019-01-18_12_15_41_282062 | 14:56 |
| *** jtomasek has quit IRC | 14:56 | |
| quiquell | jfrancoa: Will look at it | 14:56 |
| *** saneax has quit IRC | 14:56 | |
| *** jtomasek has joined #oooq | 14:59 | |
| *** jtomasek has quit IRC | 15:00 | |
| *** jtomasek has joined #oooq | 15:01 | |
| chandan_kumar | weshay: sure | 15:05 |
| *** dsneddon has joined #oooq | 15:06 | |
| *** bogdando has joined #oooq | 15:08 | |
| weshay | chandan_kumar, thanks | 15:09 |
| weshay | panda, ok.. ready now.. sorry for the delay | 15:09 |
| weshay | panda, should just be a few minutes | 15:09 |
| *** dsneddon has quit IRC | 15:10 | |
| quiquell | jfrancoa: target_upgrade_verisions is added by the upgrade releases files like master-undercloud-rocky-overcoud.yaml isn't it ? | 15:17 |
| quiquell | jfrancoa: Maybe there are changes there | 15:17 |
| chandan_kumar | weshay: https://github.com/openstack/octavia-tempest-plugin/commit/0a0f9b342a73c924ba9e7142878a2889ac2061aa broke it. | 15:17 |
| chandan_kumar | and https://github.com/openstack/octavia-tempest-plugin/commit/0a0f9b342a73c924ba9e7142878a2889ac2061aa#diff-b03119484154d807139512ef9393b286R28 -> adds a new import octavia_lib comes from https://github.com/openstack/octavia-lib which is not yet packaged in RDO. | 15:17 |
| chandan_kumar | Best solution pin the plugin in rdo at known commit and till then work on packaging the dependence in RDO. | 15:17 |
| chandan_kumar | Once package is available add a Requires in python-octavia-tests-tempest packages. | 15:17 |
| weshay | chandan_kumar, sounds good to me | 15:19 |
| quiquell | panda, marios, weshay: We need this libvirt changes to be merged for the repro https://review.openstack.org/#/c/629839/ | 15:19 |
| chandan_kumar | weshay: I am adding a temprory workaround in rodinfo | 15:20 |
| marios | quiquell: added to next reviews list | 15:20 |
| quiquell | rlandy: Even if we don't use ansible-galaxy at the script is a little effort to make the role galaxy friendly https://review.rdoproject.org/r/#/c/18397/ | 15:20 |
| quiquell | rlandy: just need some meta | 15:20 |
| quiquell | marios: thanks | 15:20 |
| marios | quiquell: did you see my comment on the review does it need to match the repo name? | 15:21 |
| marios | quiquell: https://review.rdoproject.org/r/#/c/18397/ i mean | 15:21 |
| rlandy | panda: need your collect logs expertise | 15:22 |
| jfrancoa | quiquell: I think this patch (I took it from the some time ago) should fix the issue https://review.openstack.org/#/c/607525/ | 15:22 |
| quiquell | jfrancoa: yep | 15:23 |
| quiquell | marios: let me check | 15:23 |
| ssbarnea|rover | generic question regarding reqs: is ok to keep loading test-reqs.txt with reqs that are needed in a single environment? this kinda bloats for no reason the other environments. | 15:23 |
| jfrancoa | quiquell: I think that the fs01 doesn't use the master-undercloud-rocky-overcloud release file anymore, it performs now a master to master upgrade. So the target_upgrade_version is not defined in master.yaml release file | 15:24 |
| ssbarnea|rover | this is usually avoiding by listing these in tox.ini on that specific env. is this ok or not really? | 15:24 |
| quiquell | marios: answered | 15:25 |
| rlandy | panda: I need to access releases.sh from undercloud | 15:25 |
| quiquell | jfrancoa: then we really need your review | 15:25 |
| rlandy | quiquell: re: ansible-galaxy = good for next sprint | 15:25 |
| quiquell | rlandy: but we can add the meta now, is not a big deal | 15:25 |
| rlandy | panda: can you join blue for a few minutes? | 15:25 |
| ssbarnea|rover | rlandy: ansible-galaxy is DEPRECATED | 15:25 |
| marios | quiquell: k thanks | 15:25 |
| quiquell | ssbarnea|rover: hummm really ? | 15:26 |
| rlandy | it's not required either way | 15:26 |
| quiquell | ssbarnea|rover: I though it was like the proper way to install roles and express dependencies | 15:26 |
| ssbarnea|rover | not even accepting bugs. the new silver bulet is called mazer | 15:26 |
| rlandy | we can work just fine - with three lines of code | 15:26 |
| quiquell | ssbarnea|rover: but mazer is very alpha | 15:26 |
| quiquell | rlandy: ack | 15:26 |
| jfrancoa | quiquell: should I open a lp for the issue? | 15:26 |
| rlandy | I would prefer not to throw everything branch new | 15:26 |
| rlandy | brand new | 15:26 |
| quiquell | jfrancoa: yep ssbarnea|rover can you take a look ? | 15:26 |
| rlandy | and break stuff we don;t need to fix | 15:26 |
| ssbarnea|rover | quiquell: you didn't read the news, bugs are closed on ansible-galaxy with wont fix. this is a no-go. | 15:27 |
| chandan_kumar | weshay: https://review.rdoproject.org/r/#/c/18401/ | 15:27 |
| quiquell | rlandy: sure sure fair enough | 15:27 |
| rlandy | MVP | 15:27 |
| quiquell | ssbarnea|rover: but mazer repo say it's very WIP so we are in limbo ? | 15:27 |
| quiquell | ssbarnea|rover: reading it from here https://github.com/ansible/mazer | 15:27 |
| ssbarnea|rover | quiquell: we should try it, i think they did not bother to update the msg. | 15:27 |
| quiquell | ssbarnea|rover: cool, because I like it better than ansible-galaxy is more powerful | 15:28 |
| ssbarnea|rover | i know nothing about mazer other than getting my bugs closed on galaxy with "deprecation" message. | 15:28 |
| ssbarnea|rover | so the msg is clear. we either use mazer or wait for it to mature. lets see how alpha it is next sprint. | 15:29 |
| quiquell | ssbarnea|rover: ack... next sprint then, but lake make the repro role a proper usable role | 15:29 |
| quiquell | read you next week | 15:30 |
| *** quiquell is now known as quiquell|off | 15:30 | |
| ssbarnea|rover | quiquell: rlandy on the other hand i managed to bring molecule change in a very good shape, two jobs c7/f28,... https://review.openstack.org/#/c/613672/ | 15:30 |
| ssbarnea|rover | we should start testing more roles with it | 15:31 |
| jrist | ssbarnea|rover: Aborting, target uses selinux but python bindings (libselinux-python) aren't installed! :( | 15:31 |
| jrist | I'm going to try a yum -y update on the centos7 box | 15:31 |
| ssbarnea|rover | jrist: something is really weird because i removed them on mysystem and run qs, and it detected them as missing, installed and continued. | 15:32 |
| jrist | *boggle* | 15:33 |
| jrist | ssbarnea|rover: I installed them on both the virthost and the machine I'm running quickstart from | 15:33 |
| ssbarnea|rover | jrist: but in my case VIRTHOST=127.0.0.2 -- didn't had time to test with remote one, maybe this is the issue. | 15:33 |
| weshay | panda, https://review.rdoproject.org/r/18402 | 15:33 |
| panda | rlandy: shoot | 15:34 |
| jrist | ssbarnea|rover: I can try local. where do you start local? from stack user or just any user? | 15:35 |
| ssbarnea|rover | jrist: well, in my case i was /root :p | 15:35 |
| weshay | chandan_kumar++ | 15:35 |
| hubbot1 | weshay: chandan_kumar's karma is now 1 | 15:35 |
| weshay | what? | 15:35 |
| weshay | chandan_kumar++ | 15:35 |
| hubbot1 | weshay: chandan_kumar's karma is now 2 | 15:35 |
| weshay | chandan_kumar++ | 15:35 |
| hubbot1 | weshay: chandan_kumar's karma is now 3 | 15:35 |
| weshay | chandan_kumar++ | 15:35 |
| hubbot1 | weshay: chandan_kumar's karma is now 4 | 15:35 |
| weshay | chandan_kumar++ | 15:35 |
| hubbot1 | weshay: chandan_kumar's karma is now 5 | 15:35 |
| weshay | that's better | 15:35 |
| panda | %karma chandankumar | 15:36 |
| hubbot1 | panda: Karma for "chandankumar" has been increased 6 times and decreased 1 time for a total karma of 5. | 15:36 |
| panda | %karma chkumar | 15:36 |
| hubbot1 | panda: chkumar has neutral karma. | 15:36 |
| rlandy | panda: can you join my blue for five minute - will be quicker? | 15:36 |
| weshay | thanks for the help panda | 15:36 |
| weshay | panda+ | 15:36 |
| weshay | panda++ | 15:37 |
| hubbot1 | weshay: panda's karma is now 13 | 15:37 |
| weshay | ssbarnea|rover, ok.. I think the dlrn_api snafu is now totally resolved | 15:37 |
| jrist | lol weshay | 15:37 |
| weshay | jrist, howdy | 15:37 |
| jrist | ssbarnea|rover: tisk! | 15:37 |
| jrist | weshay++ | 15:37 |
| hubbot1 | jrist: weshay's karma is now 13 | 15:37 |
| jrist | ssbarnea++ | 15:37 |
| * jrist glare hubbot1 | 15:37 | |
| jrist | ssbarnea|rover: possible it needs a reboot? (I wouldn't think so, but who knwos) | 15:38 |
| ssbarnea|rover | jrist: please keep me updated with findings, once we narrow down the issue, i will try to add a fix. | 15:38 |
| panda | rlandy: sure | 15:39 |
| weshay | jrist, you don't have python2-libselinux on the box? | 15:39 |
| panda | rlandy: id ? | 15:39 |
| ssbarnea|rover | jrist: afaik selinux needs reboot, also centos 7.5->7.6 would be another reasons. also try not to make too many changes at once before re-running as i want to know what was the reason. | 15:39 |
| jrist | weshay: I do. | 15:39 |
| rlandy | panda: https://bluejeans.com/1167276556 | 15:39 |
| jrist | ssbarnea|rover: only change now is the 7.6 change | 15:39 |
| jrist | oh, and a reboot due to that. | 15:40 |
| jrist | should have tried a reboot first! | 15:40 |
| ssbarnea|rover | weshay: is not that, that is the error, even after installing libselinux. likely he is running code from a venv (or machines) where selinux bindings are not available | 15:40 |
| weshay | https://github.com/openstack/tripleo-quickstart/blob/master/quickstart.sh#L114-L149 | 15:40 |
| weshay | ssbarnea|rover, oh ya.. | 15:41 |
| weshay | I thought we allowed global modules as well | 15:41 |
| jrist | we should, since I installed via yum/dnf | 15:41 |
| weshay | jrist, aye | 15:41 |
| weshay | jrist, :) https://dmsimard.com/2016/01/08/selinux-python-virtualenv-chroot-and-ansible-dont-play-nice/ | 15:42 |
| ssbarnea|rover | weshay: is happening much later, inside tripleo-inventory role. i bet that he can do "python import selinux" inside qs env. btw, try. | 15:42 |
| *** dsneddon has joined #oooq | 15:42 | |
| jrist | weshay: I should have guessed | 15:43 |
| weshay | lolz | 15:43 |
| ssbarnea|rover | when you do: venv: ansible "...delegate_localhost: .... " you can discover that ansible will not use the same python interpreter as you used to call ansible. | 15:43 |
| ssbarnea|rover | mainly, you escape the venv (or endup in another one). | 15:45 |
| jrist | so | 15:45 |
| jrist | somehow I interrupt the process | 15:45 |
| jrist | and go into the venv | 15:45 |
| jrist | during tripleo-inventory? | 15:45 |
| jrist | and do the python import selinux?! | 15:45 |
| ssbarnea|rover | yeah, you could do that, or just do "source $WORKSPACE/bin/activate" and test the python import | 15:46 |
| *** dsneddon has quit IRC | 15:46 | |
| ssbarnea|rover | which python should report one in your workspace, and the python import selinux should still work. | 15:47 |
| jrist | on the virthost or locally? | 15:48 |
| *** udesale has quit IRC | 15:48 | |
| ssbarnea|rover | local, btw, can you past a git with a little bit more console log? i want to see 4-5 ansible tasks before the failure. | 15:49 |
| jrist | well I just started a new one. | 15:49 |
| jrist | ssbarnea|rover: post 7.6 reboot | 15:50 |
| chandan_kumar | hubbot1++ | 15:50 |
| hubbot1 | chandan_kumar: hubbot1's karma is now 3 | 15:50 |
| jrist | ssbarnea|rover: ah good. still failing. :) | 15:52 |
| jrist | at least it's repeatable! | 15:52 |
| rlandy | panda: https://logs.rdoproject.org/67/631067/7/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-pike-branch/92a0edb/logs/quickstart_collect_logs.log | 15:53 |
| jrist | ssbarnea|rover: all the pastes incoming | 15:54 |
| jrist | ssbarnea|rover: no ansible output style on paste.openstack.org? :) | 15:56 |
| jrist | http://paste.openstack.org/show/742967/ | 15:56 |
| jrist | ssbarnea|rover: ^ | 15:56 |
| weshay | jrist, can you tmate for us? | 15:57 |
| * jrist glare weshay | 15:57 | |
| jrist | actually I don't have it installed | 15:57 |
| jrist | (new box) | 15:57 |
| weshay | why you glarin | 15:57 |
| weshay | biatch :) | 15:57 |
| jrist | installing | 15:58 |
| weshay | jrist, tmate from your laptop, ssh to new box | 15:58 |
| weshay | jeebus | 15:58 |
| jrist | ssh WnZ3JhOciAMgCShn6fmIyqF8K@sf2.tmate.io | 15:58 |
| weshay | thanks | 15:58 |
| jrist | that work^ ? | 15:58 |
| weshay | jrist, ssbarnea|rover https://bluejeans.com/u/whayutin/ | 15:59 |
| jrist | weshay: what's your number for bj | 15:59 |
| weshay | https://bluejeans.com/4113567798 | 15:59 |
| *** dsneddon has joined #oooq | 16:00 | |
| weshay | ssbarnea|rover, can you join? | 16:00 |
| *** jfrancoa has quit IRC | 16:10 | |
| weshay | ssbarnea|rover, where did you go my friend? | 16:13 |
| chandan_kumar | weshay: https://review.rdoproject.org/r/#/c/18401/ is merged now | 16:16 |
| * chandan_kumar hides and see ya on monday | 16:17 | |
| *** chandan_kumar is now known as codemonster | 16:17 | |
| weshay | thank you!! | 16:17 |
| jrist | weshay: https://bugs.launchpad.net/tripleo/+bug/1812324 | 16:18 |
| openstack | Launchpad bug 1812324 in tripleo "Quickstart inventory fails saying packages are not installed" [Low,Incomplete] - Assigned to Sorin Sbarnea (ssbarnea) | 16:18 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-quickstart-extras-gate-master-delorean-full-featureset052 @ (1 more message) | 16:32 |
| weshay | jrist, https://github.com/openstack/tripleo-quickstart/blob/master/quickstart.sh#L114-L149 | 16:33 |
| weshay | ok.. putting up a patch | 16:33 |
| *** zul has joined #oooq | 16:35 | |
| ssbarnea|rover | reminder: in less than an hour upstream gerrit will go down for upgrade. | 16:38 |
| *** rlandy is now known as rlandy|brb | 16:39 | |
| weshay | ssbarnea|rover, thanks! hey.. chat w/ me for a sec | 16:41 |
| weshay | jrist, ssbarnea|rover https://review.openstack.org/#/c/631825/ | 16:41 |
| ssbarnea|rover | weshay: nice one! btw out of curiosity how it comes that it was using python3 on centos7? | 16:42 |
| *** dtrainor has quit IRC | 16:42 | |
| weshay | ssbarnea|rover, chat w/ me on blue for a hot sec | 16:42 |
| ssbarnea|rover | jrist: have you done some special change to force use of py3? | 16:42 |
| weshay | if you can.. if not we'll do it later | 16:43 |
| jrist | neg | 16:43 |
| ssbarnea|rover | sure joining now. | 16:43 |
| weshay | ssbarnea|rover, thanks | 16:43 |
| *** dtrainor has joined #oooq | 16:43 | |
| weshay | ssbarnea|rover, https://trunk.rdoproject.org/centos7-master/wes/ | 16:45 |
| *** bogdando has quit IRC | 16:47 | |
| jrist | ssbarnea|rover: looks like it's working. currently installing undercloud | 16:50 |
| jrist | weshay: ^ | 16:50 |
| weshay | ya.. I'm on it | 16:50 |
| weshay | so both python2 and python3 are install, ansible picked python2 | 16:51 |
| ssbarnea|rover | weshay: https://review.openstack.org/#/c/627880/ | 16:56 |
| *** kopecmartin is now known as kopecmartin|off | 16:57 | |
| *** panda is now known as panda|off | 17:01 | |
| *** jpena is now known as jpena|away | 17:01 | |
| *** jpena|away is now known as jpena|off | 17:01 | |
| *** dsneddon has quit IRC | 17:07 | |
| *** bogdando has joined #oooq | 17:08 | |
| *** bogdando has quit IRC | 17:08 | |
| *** codemonster is now known as chandankumar | 17:08 | |
| *** dsneddon has joined #oooq | 17:13 | |
| *** dsneddon has quit IRC | 17:19 | |
| *** jbadiapa has quit IRC | 17:24 | |
| *** creepy_owlet is now known as dtantsur|afk | 17:38 | |
| *** rlandy|brb is now known as rlandy | 17:48 | |
| *** dsneddon has joined #oooq | 17:53 | |
| *** trown is now known as trown|lunch | 17:56 | |
| *** dsneddon has quit IRC | 17:57 | |
| *** derekh has quit IRC | 18:00 | |
| jaosorior | hey folks! a review for this would be appreciated https://review.openstack.org/#/c/631758/ | 18:01 |
| *** ykarel|away has joined #oooq | 18:04 | |
| *** dsneddon has joined #oooq | 18:11 | |
| *** jaosorior has quit IRC | 18:13 | |
| *** ccamacho has quit IRC | 18:14 | |
| rlandy | panda|off: quiquell|off: ok - I'm a little off with spacing but here is our new reproducer script ... https://logs.rdoproject.org/67/631067/8/openstack-check/tripleo-ci-centos-7-multinode-1ctlr-featureset010/b70080a/logs/reproducer-zuul-based-quickstart.sh | 18:14 |
| panda|off | \0/ | 18:15 |
| *** dsneddon has quit IRC | 18:16 | |
| rlandy | panda|off: thanks for your help - data was captured | 18:20 |
| *** dsneddon has joined #oooq | 18:22 | |
| ssbarnea|rover | jrist: can you confirm ifhttps://review.openstack.org/#/c/631825/ is fixing your issue please? | 18:28 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-quickstart-extras-gate-master-delorean-full-featureset052 @ (1 more message) | 18:32 |
| weshay | rlandy, thinking we can just merge this https://review.rdoproject.org/r/#/c/18403/ | 18:34 |
| weshay | rlandy, is https://review.rdoproject.org/r/#/c/18403/ ready for me to poke at? | 18:34 |
| ssbarnea|rover | weshay: you asked about testing with fedora-latest? you got it https://review.openstack.org/#/c/613672/31 | 18:34 |
| * rlandy checks | 18:34 | |
| weshay | ssbarnea|rover, ah cool | 18:35 |
| *** ykarel|away has quit IRC | 18:35 | |
| weshay | ssbarnea|rover, here right? https://review.openstack.org/#/c/613672/30..31/tox.ini | 18:35 |
| rlandy | weshay: will merge https://review.rdoproject.org/r/#/c/18403 after CI runs | 18:35 |
| weshay | k | 18:35 |
| rlandy | weshay: is which review ready for you to poke at? Did you mean https://review.rdoproject.org/r/#/c/18403/? | 18:36 |
| ssbarnea|rover | weshay: yep, as easy as it appears. i didn't add job, two are enough for start. right? | 18:36 |
| rlandy | that one is your review | 18:36 |
| weshay | oh sorry | 18:36 |
| weshay | rlandy, https://logs.rdoproject.org/67/631067/8/openstack-check/tripleo-ci-centos-7-multinode-1ctlr-featureset010/b70080a/logs/reproducer-zuul-based-quickstart.sh | 18:37 |
| weshay | ssbarnea|rover, ya.. totally | 18:38 |
| rlandy | weshay: my spacing is a bit off | 18:38 |
| weshay | meh | 18:38 |
| rlandy | I'm fixing that | 18:39 |
| rlandy | see what I mean? | 18:39 |
| rlandy | fixing that | 18:39 |
| rlandy | the info is capture ok - so that's good | 18:39 |
| weshay | ssbarnea|rover, so to start.. we're just covering cross platform --install-deps ya? | 18:40 |
| weshay | ssbarnea|rover, looks good to me | 18:41 |
| weshay | good start | 18:42 |
| weshay | ah ya. .w/ export | 18:49 |
| *** zul has quit IRC | 19:03 | |
| weshay | ssbarnea|rover, switch https://review.openstack.org/#/c/613672/31/zuul.d/layout.yaml voting to false for both jobs | 19:12 |
| weshay | and we'll merge it up I think | 19:12 |
| weshay | everything starts as non-voting | 19:13 |
| ssbarnea|rover | weshay: done. hopefully it will not get ignored. i seen very few failures so far. | 19:14 |
| ssbarnea|rover | weshay: revote. i fixed it. | 19:16 |
| *** holser_ has quit IRC | 19:19 | |
| *** trown|lunch is now known as trown | 19:21 | |
| rlandy | hmmm ... someone who know releases stuff ... | 19:29 |
| rlandy | ssbarnea|rover; or weshay ... if we defined releases.sh *and* dlrn_hash_tag - is that a problem? | 19:31 |
| * rlandy thinks not | 19:34 | |
| ssbarnea|rover | rlandy: not sure if why but git is prompting me for user while trying to clone github. I would have prefered ssh approach to avoid having to login to gihub. | 19:59 |
| ssbarnea|rover | it may be a local issue. | 19:59 |
| rlandy | are you using the latest reviews | 20:01 |
| rlandy | I think quique is still working there | 20:01 |
| ssbarnea|rover | rlandy: i tried https://logs.rdoproject.org/67/631067/8/openstack-check/tripleo-ci-centos-7-multinode-1ctlr-featureset010/b70080a/logs/reproducer-zuul-based-quickstart.sh | 20:01 |
| ssbarnea|rover | which is doing a clone using https | 20:01 |
| rlandy | in the process of reinstalling | 20:01 |
| ssbarnea|rover | even worse, trying to login failed. | 20:01 |
| rlandy | git clone https://github.com/$REPO? | 20:01 |
| rlandy | that line? | 20:02 |
| ssbarnea|rover | yes. | 20:02 |
| rlandy | are you using libvirt? | 20:02 |
| rlandy | or only cloning ansible-role-tripleo-ci-reproducer? | 20:02 |
| ssbarnea|rover | rlandy: yes, LIBVIRT=1 is in my user profile | 20:02 |
| ssbarnea|rover | but is unrelated to that issue. | 20:03 |
| rlandy | hmm which part is asking for your user? | 20:03 |
| ssbarnea|rover | + git clone https://github.com/rdoinfra/ansible-role-tripleo-ci-reproducer | 20:03 |
| rlandy | yeah | 20:03 |
| rlandy | if you try cone that manually the firs time, does it ask for user ask well? | 20:03 |
| ssbarnea|rover | is missing a dash! | 20:03 |
| ssbarnea|rover | rdo-infra! is the org | 20:04 |
| rlandy | rdo-infra | 20:04 |
| rlandy | sorry | 20:04 |
| rlandy | fixing | 20:04 |
| rlandy | ssbarnea|rover: fix submitted - sorry about that | 20:05 |
| rlandy | ssbarnea|rover: you'll also need the latest fixes to the jinja | 20:09 |
| ssbarnea|rover | rlandy: nopb, just tell me where to get them from. | 20:09 |
| ssbarnea|rover | anyone having a online installer for tmate on centos? | 20:10 |
| rlandy | https://review.openstack.org/#/c/631067/ | 20:10 |
| rlandy | latest change | 20:10 |
| rlandy | this is all still testing | 20:10 |
| rlandy | so user beware | 20:10 |
| ssbarnea|rover | rlandy: you are not reformarting my host right? even so i have baremetal re-provisioning configured :D | 20:11 |
| rlandy | ssbarnea|rover: I will quote weshay when he said we can consider a blank system | 20:12 |
| * rlandy suggested that developer may not want their systems destroyed | 20:12 | |
| rlandy | and was told not to worry about it | 20:12 |
| ssbarnea|rover | when the time comes i will test with it after reprovisioning. | 20:13 |
| rlandy | ssbarnea|rover: when I tested libvirt originally, I did it with the old role | 20:13 |
| rlandy | so I am tested the new role now as well | 20:13 |
| rlandy | I need to update my test box to f28 now | 20:13 |
| rlandy | I kept it on f27 as I had a working system | 20:14 |
| rlandy | but time to start again | 20:14 |
| ssbarnea|rover | i can give you access to my f28 box so you can play with it if you want | 20:14 |
| ssbarnea|rover | rlandy: btw, how am I supposed to test the new changes? i guess I need to wait for zuul to run to produce the script? or is another way? | 20:15 |
| rlandy | ssbarnea|rover: manually copy and edit | 20:15 |
| rlandy | so that your old script | 20:16 |
| rlandy | and fix the formatting | 20:16 |
| rlandy | change depends-on to review | 20:16 |
| rlandy | make sure the release.sh file is spaced right | 20:16 |
| rlandy | you're using my first attempt | 20:16 |
| ssbarnea|rover | few more small bits to fix https://seashells.io/v/dbVAAdTv | 20:17 |
| rlandy | k - fixing | 20:18 |
| ssbarnea|rover | you need to make repo_list a bash array to work with it, like in https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays | 20:18 |
| ssbarnea|rover | clearly not as nice as python | 20:19 |
| rlandy | ssbarnea|rover: we could just do this all in an ansible role | 20:20 |
| ssbarnea|rover | rlandy: a hint, maybe you can write a small script that produce a testable reprocerscript directly from the codebase. like "tox -e re" which | 20:20 |
| rlandy | and forget the bash | 20:20 |
| ssbarnea|rover | it would make much easier to change the jinja2 template and rerun it | 20:21 |
| rlandy | I might just do that | 20:21 |
| rlandy | all I really need to do is create a playbook | 20:22 |
| rlandy | which is pretty much what this does | 20:22 |
| rlandy | ssbarnea|rover: really - the only complication is whether a user wants libvirt or not | 20:26 |
| rlandy | otherwise all you need to do is run pre and launcher | 20:26 |
| rlandy | what we need from the job is the settings yaml fuke | 20:27 |
| rlandy | file | 20:27 |
| weshay | fuke :) | 20:27 |
| ssbarnea|rover | nice name for a new tool | 20:27 |
| ssbarnea|rover | i am almost crying ... | 20:28 |
| weshay | ssbarnea|rover, keep in mind w/ the reproducer we're shooting for a mvp | 20:28 |
| rlandy | weshay: everyone dislikes the bash | 20:29 |
| rlandy | which all we are doing is cloning two repos and setting up the ansible env | 20:29 |
| rlandy | we would just create the yaml settings file | 20:29 |
| rlandy | and let users figure life out on their own | 20:29 |
| rlandy | adding python to the mix is not helpful | 20:30 |
| rlandy | we just go with another ansible playbook | 20:30 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/560445, (1 more message) | 20:32 |
| *** kopecmartin|off has quit IRC | 20:33 | |
| *** kopecmartin has joined #oooq | 20:36 | |
| *** rlandy has quit IRC | 21:25 | |
| *** agopi has quit IRC | 21:41 | |
| *** trown is now known as trown|outtypewww | 21:47 | |
| *** jtomasek has quit IRC | 22:00 | |
| jrist | ssbarnea|rover: yes, that worked | 22:11 |
| *** holser_ has joined #oooq | 22:22 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/queens: tripleo- (1 more message) | 22:32 |
| *** holser_ has quit IRC | 23:47 | |
| *** tosky has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!