| *** threestrands has joined #openstack-ironic | 00:00 | |
| openstackgerrit | Verification of a change to openstack/ironic failed: Extend timeout on CI job with automated cleaning https://review.opendev.org/720065 | 00:02 |
|---|---|---|
| *** cdearborn has quit IRC | 00:23 | |
| *** ricolin has joined #openstack-ironic | 01:13 | |
| *** ricolin_ has joined #openstack-ironic | 01:19 | |
| *** ricolin has quit IRC | 01:21 | |
| *** mraineri has joined #openstack-ironic | 01:26 | |
| *** Goneri has quit IRC | 01:37 | |
| openstackgerrit | zhufl proposed openstack/ironic master: Fix missing print format in log messages https://review.opendev.org/720114 | 01:41 |
| *** Qianbiao has joined #openstack-ironic | 01:50 | |
| Qianbiao | morning ironic. | 01:54 |
| *** ociuhandu has joined #openstack-ironic | 01:56 | |
| *** mkrai has joined #openstack-ironic | 02:07 | |
| *** ociuhandu has quit IRC | 02:13 | |
| *** k_mouza has joined #openstack-ironic | 02:23 | |
| *** k_mouza has quit IRC | 02:27 | |
| *** tzumainn has quit IRC | 02:51 | |
| *** mkrai has quit IRC | 03:23 | |
| *** mkrai has joined #openstack-ironic | 03:24 | |
| *** hamzy_ has joined #openstack-ironic | 03:29 | |
| *** hamzy has quit IRC | 03:31 | |
| *** mkrai has quit IRC | 04:05 | |
| *** mkrai_ has joined #openstack-ironic | 04:06 | |
| *** gyee has quit IRC | 04:13 | |
| *** mkrai has joined #openstack-ironic | 04:14 | |
| *** ociuhandu has joined #openstack-ironic | 04:15 | |
| *** mkrai_ has quit IRC | 04:17 | |
| *** ociuhandu has quit IRC | 04:33 | |
| *** mkrai has quit IRC | 05:01 | |
| *** Qianbiao has quit IRC | 05:05 | |
| *** mkrai has joined #openstack-ironic | 05:06 | |
| *** dhellmann has quit IRC | 05:16 | |
| openstackgerrit | Steve Baker proposed openstack/ironic master: Centralise imports of wsme types https://review.opendev.org/703898 | 05:28 |
| openstackgerrit | Steve Baker proposed openstack/ironic master: Don't use wsme test webapp for patch tests https://review.opendev.org/704485 | 05:28 |
| openstackgerrit | Steve Baker proposed openstack/ironic master: Do all serialization in the expose decorator https://review.opendev.org/704486 | 05:28 |
| openstackgerrit | Steve Baker proposed openstack/ironic master: Add validate decorator to expose https://review.opendev.org/704487 | 05:28 |
| openstackgerrit | Steve Baker proposed openstack/ironic master: WIP Add function definition handling https://review.opendev.org/704488 | 05:28 |
| openstackgerrit | Steve Baker proposed openstack/ironic master: WIP Add json parsing to args https://review.opendev.org/704489 | 05:28 |
| openstackgerrit | Steve Baker proposed openstack/ironic master: WIP Add wsme core types, remove WSME https://review.opendev.org/704490 | 05:28 |
| *** dhellmann has joined #openstack-ironic | 05:31 | |
| *** dhellmann has quit IRC | 05:38 | |
| *** ociuhandu has joined #openstack-ironic | 05:39 | |
| *** dhellmann has joined #openstack-ironic | 05:45 | |
| *** ociuhandu has quit IRC | 05:51 | |
| *** threestrands has quit IRC | 05:54 | |
| *** dhellmann has quit IRC | 05:58 | |
| *** Qianbiao has joined #openstack-ironic | 05:59 | |
| *** dhellmann has joined #openstack-ironic | 06:13 | |
| *** ociuhandu has joined #openstack-ironic | 06:26 | |
| *** dhellmann has quit IRC | 06:34 | |
| *** Qianbiao is now known as Qianbiao|afk | 06:36 | |
| *** bdodd has quit IRC | 06:36 | |
| *** bdodd has joined #openstack-ironic | 06:39 | |
| *** dhellmann has joined #openstack-ironic | 06:40 | |
| iurygregory | good morning Ironic! | 06:41 |
| openstackgerrit | Harald Jensås proposed openstack/ironic-inspector master: Add BasePhysnetHook https://review.opendev.org/720010 | 06:45 |
| openstackgerrit | Harald Jensås proposed openstack/ironic-inspector master: Port physical network hook https://review.opendev.org/718594 | 06:45 |
| arne_wiebalck | Good morning iurygregory and ironic! | 06:53 |
| iurygregory | arne_wiebalck, o/ | 07:05 |
| *** Qianbiao|afk is now known as Qianbiao | 07:09 | |
| Qianbiao | hello ironic. | 07:09 |
| Qianbiao | hello, when deploy os to baremetal, an error occurs: openstack:85c78981-f85a-49e1-9c67-0b3cb93dfc4c-lun-1 bs=1M oflag=direct (no filter matched) | 07:10 |
| Qianbiao | any hints? | 07:10 |
| Qianbiao | log details: http://paste.openstack.org/show/792137/ | 07:11 |
| *** rpittau|afk is now known as rpittau | 07:28 | |
| rpittau | good morning ironic! o/ | 07:28 |
| *** iurygregory has quit IRC | 07:31 | |
| *** iurygregory has joined #openstack-ironic | 07:32 | |
| iurygregory | morning rpittau o/ | 07:33 |
| rpittau | hey iurygregory :) | 07:33 |
| *** ociuhandu has quit IRC | 07:40 | |
| *** ociuhandu has joined #openstack-ironic | 07:41 | |
| iurygregory | Qianbiao, to me it seems it couldn't configure the node, do you have some conductors and ipa logs ? | 07:41 |
| iurygregory | maybe you are using a wrong partition | 07:42 |
| Qianbiao | <iurygregory> log details: http://paste.openstack.org/show/792137/ | 07:43 |
| Qianbiao | your name contains a lot of wierd chars when copy | 07:43 |
| iurygregory | Qianbiao, I was looking at this link, but this is not enough I would say | 07:43 |
| Qianbiao | i see, there are only those useful log in conductor | 07:44 |
| iurygregory | conductor, ipa | 07:44 |
| iurygregory | more details are always welcome =) | 07:44 |
| Qianbiao | it seems this path does not exists: /var/lib/ironic/http/images/85c78981-f85a-49e1-9c67-0b3cb93dfc4c/disk | 07:44 |
| Qianbiao | and instead it has : 7a91f5a4-12f8-4886-8da0-5e08c98354ce ubuntu-xenial-16.04.meta ubuntu-xenial-16.04.qcow2 | 07:45 |
| Qianbiao | not sure why it got a wrong id | 07:45 |
| iurygregory | hummm | 07:45 |
| iurygregory | yeah | 07:45 |
| iurygregory | you will need to try again .-. | 07:46 |
| *** ociuhandu has quit IRC | 07:46 | |
| Qianbiao | sure will figure it out | 07:46 |
| iurygregory | good luck! | 07:47 |
| iurygregory | arne_wiebalck, rpittau if a partition is found in /dev/sda the deploy logs should show this partition in the lsblk output right? | 07:52 |
| rpittau | iurygregory: yes, remind me the exact command we're using | 07:55 |
| iurygregory | let me find here | 07:55 |
| iurygregory | Im puzzled since uefi partition is found in sda15 but lsblk doesn't show O.o | 07:56 |
| openstackgerrit | Verification of a change to openstack/ironic failed: Extend timeout on CI job with automated cleaning https://review.opendev.org/720065 | 07:56 |
| iurygregory | rpittau, https://github.com/openstack/ironic-python-agent/blob/3da356de257e10ddbad1f84054d1d2357d152941/ironic_python_agent/utils.py#L67 | 07:57 |
| openstackgerrit | Riccardo Pittau proposed openstack/networking-baremetal master: BUILD_TIMEOUT is not needed https://review.opendev.org/720016 | 07:57 |
| iurygregory | In the logs for wholedisk http://paste.openstack.org/show/792141/ | 07:58 |
| arne_wiebalck | iurygregory: this is not a partition which comes with the image, is it? Since the lsblk output is from before the image is dumped. | 07:58 |
| arne_wiebalck | I would think :) | 07:58 |
| iurygregory | it's from before ? O.o even if we only get the deploy logs after ipa runs? | 07:59 |
| rpittau | yeah, that depends on when the command runs | 07:59 |
| iurygregory | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_240/714453/12/check/ironic-tempest-ipa-wholedisk-uefi-ipmi-iscsi-tinyipa/2402d0c/controller/logs/ironic-bm-logs/node-0_console_2020-04-14-14%3A45%3A49_log.txt | 07:59 |
| *** lucasagomes has joined #openstack-ironic | 08:14 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: [DNM] Dump info on pxeboot for uefi jobs https://review.opendev.org/719956 | 08:15 |
| openstackgerrit | Dmitry Tantsur proposed openstack/python-ironic-inspector-client master: Document the new standalone CLI https://review.opendev.org/720034 | 08:17 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: [WIP] Fix grub2 pxe job with native bionic ovmf https://review.opendev.org/716889 | 08:18 |
| *** tkajinam has quit IRC | 08:21 | |
| *** ociuhandu has joined #openstack-ironic | 08:22 | |
| openstackgerrit | Mahendra Kamble proposed openstack/ironic master: DRAC : idrac-redfish inspect updates pxe port https://review.opendev.org/693328 | 08:23 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-inspector master: Convert jobs to dib https://review.opendev.org/712516 | 08:24 |
| mgoddard | morning | 08:25 |
| mgoddard | has anyone looked at https://tinkerbell.org/ ? | 08:25 |
| iurygregory | first time i hear about it | 08:26 |
| mgoddard | might be worth a look | 08:26 |
| *** ociuhandu has quit IRC | 08:29 | |
| iurygregory | Oct 14, 2019 O.o | 08:29 |
| *** ociuhandu has joined #openstack-ironic | 08:29 | |
| iurygregory | and the other projects are very new O.o | 08:30 |
| *** ricolin_ is now known as ricolin | 08:30 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Use trailing slash in the agent command URL https://review.opendev.org/720150 | 08:30 |
| openstackgerrit | Riccardo Pittau proposed openstack/python-ironic-inspector-client master: Convert job to dib https://review.opendev.org/718698 | 08:35 |
| *** alexmcleod has joined #openstack-ironic | 08:39 | |
| *** Qianbiao has quit IRC | 08:44 | |
| *** dtantsur|afk is now known as dtantsur | 08:49 | |
| dtantsur | morning ironic, says man debugging agent tokens | 08:49 |
| *** Qianbiao has joined #openstack-ironic | 08:50 | |
| rpittau | good morning dtantsur | 08:50 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/train: Document deploy_boot_mode and boot_option for standalone deployments https://review.opendev.org/720156 | 08:55 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/train: DNM try ussuri IPA https://review.opendev.org/720157 | 08:57 |
| *** k_mouza has joined #openstack-ironic | 09:01 | |
| *** dhellmann has quit IRC | 09:05 | |
| iurygregory | morning dtantsur | 09:07 |
| *** dhellmann has joined #openstack-ironic | 09:11 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: WIP: Decompose core deploy step https://review.opendev.org/698309 | 09:22 |
| Qianbiao | hello, ironic. | 09:30 |
| Qianbiao | need help. iscsi seems could not load right image, logs: http://static.turnbig.net/ironic/ | 09:31 |
| Qianbiao | this is logs from IPA: http://static.turnbig.net/ironic/deploy/4269dbec-b05d-4471-83d5-3cb3cae8528c_2020-04-15-09-19-09.tar.gz | 09:31 |
| dtantsur | TheJulia: could you remove -2 from https://review.opendev.org/#/c/708827/ please? we need to land something to rebuild the train image | 09:34 |
| patchbot | patch 708827 - ironic-python-agent (stable/train) - Rescan after restarting the md device - 2 patch sets | 09:34 |
| dtantsur | TheJulia: also I don't think https://storyboard.openstack.org/#!/story/2007455 is fixed. It won't even reach the part that your patch is touching. | 09:39 |
| dtantsur | the fact that UEFI whole disk images don't work if the efibootmgr approach fails is.. frustrating to people | 09:42 |
| dtantsur | iurygregory: did you have any success with ^^^? | 09:43 |
| iurygregory | dtantsur, I got a strange working job... it succeeds with tempest but the node logs tell me something completely different https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_240/714453/12/check/ironic-tempest-ipa-wholedisk-uefi-ipmi-iscsi-tinyipa/2402d0c/controller/logs/ironic-bm-logs/node-0_console_2020-04-14-14%3A45%3A49_log.txt | 09:44 |
| dtantsur | WUT | 09:45 |
| iurygregory | yeah that was my reaction also =) | 09:45 |
| iurygregory | we found a efi partition in sda15 but the mount fails | 09:46 |
| dtantsur | iurygregory: the person I'm talking about reports a similar thing: bootloader fails, but the OS boots | 09:56 |
| iurygregory | wut?! =O | 09:57 |
| iurygregory | black magic | 09:57 |
| *** jamesdenton_wfh has joined #openstack-ironic | 10:10 | |
| openstackgerrit | Aija Jaunteva proposed openstack/ironic master: Update iDRAC doc about vendor passthru timeout https://review.opendev.org/720173 | 10:11 |
| openstackgerrit | Merged openstack/ironic-python-agent master: Move minimum ironic version to latest ocata https://review.opendev.org/718450 | 10:22 |
| dtantsur | iurygregory: wanna more magic? in ubuntu images the esp partition doesn't have the esp flag, like here https://opendev.org/openstack/ironic-python-agent/src/branch/master/ironic_python_agent/tests/unit/test_utils.py#L55 | 10:22 |
| dtantsur | so it's only called ESP and has only 'boot' flag | 10:22 |
| dtantsur | lovely.... | 10:22 |
| dtantsur | I wonder if we can rely on name+boot as well | 10:22 |
| *** rpittau is now known as rpittau|bbl | 10:23 | |
| dtantsur | oh, esp is an alias for boot on GPT. needs fixing. | 10:29 |
| iurygregory | OMG | 10:35 |
| iurygregory | dtantsur, I can look at this latter ( I have a session to present in 25min .-.) 5hrs of event .-. | 10:36 |
| dtantsur | iurygregory: I'm working on a patch, don't worry | 10:36 |
| iurygregory | dtantsur, tks! | 10:36 |
| * iurygregory will be afk the whole afternoon basically | 10:36 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: A boot partition on a GPT disk should be considered an EFI partition https://review.opendev.org/720184 | 10:46 |
| dtantsur | iurygregory: if you could take a quick look ^^^ would be awesome | 10:46 |
| iurygregory | dtantsur, looking now | 10:47 |
| iurygregory | dtantsur, makes sense to me! | 10:48 |
| *** derekh has joined #openstack-ironic | 10:49 | |
| *** jtomasek has quit IRC | 10:56 | |
| *** jtomasek has joined #openstack-ironic | 11:01 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: Fix the token logic to be compatible with older ironic https://review.opendev.org/720185 | 11:01 |
| dtantsur | TheJulia: ^^^ | 11:02 |
| Qianbiao | hello, may someone help look at a deployment problem, takes me a long time and could not figure out why. logs: http://paste.openstack.org/show/792146/ http://paste.openstack.org/show/792147/ | 11:02 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/train: DNM try ussuri IPA https://review.opendev.org/720157 | 11:04 |
| *** etingof is now known as etingof|brb | 11:07 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: WIP: Decompose core deploy step https://review.opendev.org/698309 | 11:08 |
| Qianbiao | Am i using a wrong IPA? the version i used is tinyipa-master. | 11:08 |
| Qianbiao | is tinyipa-master ready for ussuri | 11:08 |
| *** Qianbiao is now known as Qianbiao|afk | 11:15 | |
| *** cdearborn has joined #openstack-ironic | 11:15 | |
| *** iurygregory is now known as iurygregory_afk | 11:31 | |
| *** Qianbiao|afk has quit IRC | 11:37 | |
| *** Qianbiao|afk has joined #openstack-ironic | 11:38 | |
| dtantsur | tinyipa-master is ussuri, yes | 11:41 |
| *** ociuhandu has quit IRC | 11:42 | |
| arne_wiebalck | dtantsur: iurygregory_afk: I haven't looked at the context tbh, but will 720184 cause issues for BIOS images which use GPT? | 11:48 |
| iurygregory_afk | arne_wiebalck, when I looked at the changes it seems that we are only touching the uefi check | 11:51 |
| arne_wiebalck | iurygregory_afk: as in, we know this is UEFI and we only need to find the partition? | 11:52 |
| *** dsneddon has quit IRC | 11:53 | |
| iurygregory_afk | arne_wiebalck, as in we are only touching get_efi_part_on_device =) | 11:53 |
| iurygregory_afk | i don't think the path for bios is using this function =) | 11:53 |
| arne_wiebalck | iurygregory_afk: ok, thanks | 11:54 |
| arne_wiebalck | iurygregory_afk: I triggered on this as we have BIOS images with GPT | 11:54 |
| iurygregory_afk | arne_wiebalck, gotcha =) no worries | 11:55 |
| *** rh-jelabarre has joined #openstack-ironic | 12:07 | |
| *** etingof|brb is now known as etingof | 12:10 | |
| dtantsur | arne_wiebalck: BIOS images that use GPT use a different partition type for their partitions | 12:19 |
| dtantsur | namely, legacy_boot | 12:19 |
| dtantsur | s/their/legacy boot/ | 12:20 |
| arne_wiebalck | right, but I thought the patch will use the existence of the GPT to make assumptions about the type | 12:20 |
| dtantsur | arne_wiebalck: it does, check the updated utils | 12:20 |
| *** Goneri has joined #openstack-ironic | 12:21 | |
| arne_wiebalck | dtantsur: that was my worry :) | 12:22 |
| arne_wiebalck | dtantsur: that the sheer existence of a GPT would imply an EFI partition and hence UEFI, but the logic seems to be we are in UEFI mode, cannot find an EFI partition and now assume the one with the GPT is the one we're looking for | 12:24 |
| dtantsur | arne_wiebalck: a partition with label "boot" in a GPT table is an EFI partition, at least according to parted docs. | 12:24 |
| dtantsur | but yes, we seem to check the boot mode as well | 12:25 |
| *** ociuhandu has joined #openstack-ironic | 12:31 | |
| *** ociuhandu has quit IRC | 12:32 | |
| *** ociuhandu has joined #openstack-ironic | 12:32 | |
| arne_wiebalck | dtantsur: our images have a GPT partition table with type "BIOS boot" | 12:34 |
| dtantsur | arne_wiebalck: right, it will probably be legacy_boot in parted | 12:35 |
| dtantsur | mmmm, sorry, bios_grub | 12:35 |
| dtantsur | arne_wiebalck: see an example I got from a real deployment: https://storyboard.openstack.org/#!/story/2007455#comment-155639 | 12:36 |
| arne_wiebalck | dtantsur: "BIOS boot" is what parted displays | 12:36 |
| dtantsur | do you call it with the same arguments? could you paste the output? | 12:36 |
| arne_wiebalck | oh, these are flags, I was talking about type | 12:37 |
| *** rloo has joined #openstack-ironic | 12:37 | |
| arne_wiebalck | sure, once sec ... | 12:37 |
| dtantsur | we seem to be looking at flags | 12:37 |
| dtantsur | but it's a good idea to check if "BIOS boot" in the name will trigger an incorrect path | 12:38 |
| arne_wiebalck | sorry, I created some confusion | 12:38 |
| dtantsur | meh, our regex logic is bad indeed | 12:39 |
| dtantsur | arne_wiebalck: could you still paste your output? | 12:39 |
| arne_wiebalck | http://paste.openstack.org/show/792157/ | 12:39 |
| arne_wiebalck | I mixed fdisk and parted | 12:39 |
| arne_wiebalck | all there ^^ | 12:39 |
| arne_wiebalck | the flag is indeed bios_grub as you said | 12:40 |
| arne_wiebalck | this is on a VM, btw | 12:40 |
| arne_wiebalck | not a physical node | 12:40 |
| dtantsur | arne_wiebalck: thanks! yeah, I've realized that having "BIOS boot" in a name will trigger the incorrect logic. Thinking how to fix it... | 12:40 |
| *** yolanda has quit IRC | 12:43 | |
| dtantsur | arne_wiebalck: could you also try parted with -m flag? I'm trying to avoid parsing the human-readable representation. | 12:48 |
| arne_wiebalck | sure, one sec ... | 12:48 |
| *** rpittau|bbl is now known as rpittau | 12:49 | |
| arne_wiebalck | dtantsur: http://paste.openstack.org/show/792158/ | 12:51 |
| dtantsur | thanks! | 12:51 |
| TheJulia | good morning | 12:52 |
| dtantsur | morning TheJulia | 12:52 |
| iurygregory_afk | good morning TheJulia | 12:54 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: [WIP] A boot partition on a GPT disk should be considered an EFI partition https://review.opendev.org/720184 | 12:55 |
| dtantsur | arne_wiebalck, iurygregory_afk, this is what I ended up with ^^^ (needs unit test updates) | 12:55 |
| iurygregory_afk | dtantsur, in 5 min I will look after the session is over here o/ | 12:56 |
| * dtantsur wonders if IPA has any jobs with UEFI | 12:56 | |
| iurygregory_afk | dtantsur, I would say no .-. | 12:57 |
| iurygregory_afk | https://github.com/openstack/ironic-python-agent/blob/master/zuul.d/project.yaml | 12:57 |
| dtantsur | that's bad.. | 12:57 |
| iurygregory_afk | yeah | 12:57 |
| dtantsur | so I'm essentially shooting in darkness :) | 12:57 |
| iurygregory_afk | boom! | 12:57 |
| openstackgerrit | Merged openstack/ironic master: Extend timeout on CI job with automated cleaning https://review.opendev.org/720065 | 12:58 |
| TheJulia | dtantsur: /me blinks | 13:03 |
| TheJulia | err | 13:03 |
| * TheJulia blinks | 13:03 | |
| dtantsur | :) | 13:03 |
| dtantsur | I'd recommend against opening eyes today, but since you already did... | 13:03 |
| TheJulia | downside of changing the base ipa image, things are more unstable again | 13:04 |
| dtantsur | not entirely unexpected, unfortunately | 13:04 |
| TheJulia | yeah | 13:04 |
| TheJulia | did you ever figure out why all timeoutes in tempest really like 12 minutes? | 13:05 |
| TheJulia | err | 13:05 |
| TheJulia | really like 10 minutes when set to 12 | 13:05 |
| dtantsur | first time I hear about it | 13:05 |
| TheJulia | you've banged your head against it before | 13:05 |
| TheJulia | I remember the grumpiness of that day | 13:05 |
| TheJulia | :) | 13:05 |
| dtantsur | probably banged it too hard :D | 13:05 |
| rpittau | TheJulia: timeouts are confusing me a lot too, the variables for the tempest jobs are 3 apparently | 13:08 |
| TheJulia | yes! | 13:08 |
| TheJulia | they are :( | 13:08 |
| * TheJulia gets out lots of booze | 13:08 | |
| TheJulia | surely someone has created a beverage called "the tempest"... | 13:08 |
| * rpittau gets a big mug | 13:09 | |
| dtantsur | TheJulia: vodka with a lot of vodka and a small shot of vodka? | 13:09 |
| TheJulia | So "the tempest" is a riff on a "Dark and Stormy" | 13:10 |
| rpittau | I'm trying to figure out the issues with the dib images on some jobs that are timing out, can't see evidence of failures in the logs | 13:10 |
| TheJulia | https://www.foodandwine.com/recipes/tempest | 13:10 |
| rpittau | like this one https://zuul.opendev.org/t/openstack/build/a95f6baa700d4917a6509e4fe669e4c2 | 13:11 |
| TheJulia | yeah, so there is an outer timeout on tests, then inner step level timeouts | 13:11 |
| rpittau | mmm | 13:11 |
| openstackgerrit | Merged openstack/ironic master: Software RAID: Pass the boot mode to the IPA https://review.opendev.org/696169 | 13:12 |
| TheJulia | ohhh that job is running erase_devices | 13:15 |
| arne_wiebalck | woot | 13:15 |
| TheJulia | so that is going to need a longer timeout | 13:16 |
| TheJulia | so think 400-500 seconds for the ramdisk to be up and IPA running | 13:16 |
| *** mbeierl has quit IRC | 13:16 | |
| arne_wiebalck | thanks TheJulia and stendulker to throw 696169 under the zuul recheck bus over and over again! | 13:17 |
| TheJulia | and then think another 300+ seconds to just run erase devices in the best cases with any io pressure | 13:17 |
| TheJulia | arne_wiebalck: it was all timeout stuffs with other stuff, so happy to drive a bus over the patches a few... too many times | 13:17 |
| *** mbeierl has joined #openstack-ironic | 13:18 | |
| rpittau | sounds like frogger but with the patch instead of the frog | 13:18 |
| * TheJulia gets out the super glue to glue IRC clients to the room | 13:18 | |
| TheJulia | :) | 13:18 |
| TheJulia | rpittau: very much so | 13:18 |
| rpittau | TheJulia: I have the timout already at 2400, will try with 3600 then | 13:19 |
| TheJulia | so I think we should add more logging to our tempest plugin | 13:20 |
| TheJulia | specifically around bm wait states | 13:20 |
| TheJulia | so when the match fails or we begin to exit out of them, we actually log that | 13:20 |
| rpittau | ooohh non-standalone inspector is ok with "just" 2400 | 13:22 |
| openstackgerrit | Riccardo Pittau proposed openstack/python-ironic-inspector-client master: Convert job to dib https://review.opendev.org/718698 | 13:22 |
| TheJulia | dtantsur: commented on your comment on https://review.opendev.org/#/c/720100/1 :) | 13:23 |
| patchbot | patch 720100 - ironic-specs - WIP - Victoria Cycle Priorit(y|ies) - 1 patch set | 13:23 |
| TheJulia | food for thought | 13:23 |
| * iurygregory_afk grabs more coffee to prepare my mind for more 2hrs of programing contest | 13:24 | |
| iurygregory_afk | dtantsur, are you sharing vodka? | 13:25 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-inspector master: Convert jobs to dib https://review.opendev.org/712516 | 13:25 |
| * iurygregory_afk only saw the word =P | 13:25 | |
| iurygregory_afk | TheJulia, I added a comment for the priorities doc | 13:28 |
| dtantsur | TheJulia: running erase_devices is a no-go on virt | 13:30 |
| dtantsur | if we do it, we need to stop. it can take arbitrary amount of time, from minutes to hours | 13:30 |
| dtantsur | iurygregory_afk: vodka and programming contests? sounds appropriate :D | 13:30 |
| iurygregory_afk | dtantsur, it does! | 13:31 |
| *** dhellmann has quit IRC | 13:32 | |
| iurygregory_afk | I remember when I was learning python, the professor gave a list of questions for the class to solve, and there was 1 question that was pretty hard, one guy found the solution while we were at pub drinking | 13:33 |
| dtantsur | I'm pretty sure that's how python was invented, so fair enough | 13:34 |
| *** dhellmann has joined #openstack-ironic | 13:35 | |
| dtantsur | TheJulia: FYI https://storyboard.openstack.org/#!/story/2007557 | 13:35 |
| *** rcernin has quit IRC | 13:37 | |
| dtantsur | stable cores, please see https://review.opendev.org/#/c/718728/ | 13:46 |
| patchbot | patch 718728 - ironic-inspector (stable/stein) - [stable-only] Make ironicclient initialization fut... - 2 patch sets | 13:46 |
| *** tzumainn has joined #openstack-ironic | 13:48 | |
| etingof | what's the BM whitepaper meeting url please? | 14:00 |
| dtantsur | etingof: https://cern.zoom.us/j/95696673823 | 14:01 |
| etingof | o/ | 14:01 |
| TheJulia | should be able to join soon-ish, wrapping up a call | 14:01 |
| Qianbiao|afk | hello, can we use qcow2 img in this link directly? https://cloud-images.ubuntu.com/releases/xenial/release/ | 14:02 |
| dtantsur | Qianbiao|afk: worth giving a try. sometimes it works (e.g. with centos images). | 14:04 |
| *** Qianbiao|afk is now known as Qianbiao | 14:04 | |
| Qianbiao | Hmm, i got an error: Failed to check the number of primary partitions present on /dev/disk/by-path/ip-192.168.0.180:3260-iscsi-iqn.2008-10.org.openstack:4269dbec-b05d-4471-83d5-3cb3cae8528c-lun-1 for node 4269dbec-b05d-4471-83d5-3cb3cae8528c. | 14:04 |
| Qianbiao | may it because the qcow2 image is broken? | 14:05 |
| etingof | zoom is hopelsss | 14:05 |
| dtantsur | yeah, not the easier one to set up | 14:06 |
| dtantsur | * easiest | 14:06 |
| *** jcoufal has joined #openstack-ironic | 14:11 | |
| *** mkrai has quit IRC | 14:26 | |
| *** ociuhandu has quit IRC | 14:36 | |
| *** mkrai has joined #openstack-ironic | 14:37 | |
| rpioso | Good morning, ironic | 14:38 |
| *** mkrai_ has joined #openstack-ironic | 14:42 | |
| *** mkrai_ has quit IRC | 14:42 | |
| *** mkrai_ has joined #openstack-ironic | 14:42 | |
| *** mkrai has quit IRC | 14:42 | |
| *** tkajinam has joined #openstack-ironic | 14:43 | |
| *** ociuhandu has joined #openstack-ironic | 14:43 | |
| jungleboyj | arne_wiebalck: Sorry for being so quiet. I do still have it in my todos to read through the whole paper and make comments. :-) | 14:52 |
| arne_wiebalck | jungleboyj: thanks for joining, your feedback will be very welcome! | 14:52 |
| jungleboyj | :-) | 14:53 |
| *** mkrai_ has quit IRC | 14:54 | |
| *** mkrai_ has joined #openstack-ironic | 14:54 | |
| *** dtantsur is now known as dtantsur|brb | 14:55 | |
| *** mkrai_ has quit IRC | 14:55 | |
| TheJulia | ++ | 14:58 |
| *** priteau has joined #openstack-ironic | 15:00 | |
| * TheJulia wonders why her current cup of coffee tastes burnt | 15:02 | |
| *** gyee has joined #openstack-ironic | 15:04 | |
| rpittau | well apparently it's really just a matter of how long we want to wait https://review.opendev.org/#/c/718698/ | 15:07 |
| patchbot | patch 718698 - python-ironic-inspector-client - Convert job to dib - 3 patch sets | 15:07 |
| *** jiriprox has joined #openstack-ironic | 15:07 | |
| *** ianychoi has quit IRC | 15:13 | |
| iurygregory_afk | http://www.devstack.co.kr/ WUT O.o | 15:14 |
| TheJulia | safe? | 15:14 |
| iurygregory_afk | yup | 15:14 |
| iurygregory_afk | Matthew Treinish shared in Promoting Ironic thread | 15:15 |
| * iurygregory_afk never thought that devstack could be a name of a company XD | 15:16 | |
| rpittau | I wonder if that is "legal" | 15:16 |
| iurygregory_afk | rpittau, I had this feeling also =) | 15:16 |
| iurygregory_afk | I also hope they don't use devstack in their product lol | 15:17 |
| rpittau | also I could never use the excuse that "works in devstack" :P | 15:17 |
| iurygregory_afk | I've used this twice in my life so far | 15:17 |
| jiriprox | Hi, I would like to ask you for help. | 15:18 |
| jiriprox | I’m trying to integrate new baremetal machine with SmartNIC into our Openstack environment. | 15:19 |
| jiriprox | Is there anything what needs to be added into ramdisk with IPA which is used to boot host (with SmartNIC)? | 15:19 |
| iurygregory_afk | afik ironic has the support for it and neutron too, and you probably need is_smartnic set | 15:20 |
| iurygregory_afk | I don't think you need to change the ramdisk | 15:21 |
| rpittau | maybe specific kernel drivers? | 15:21 |
| openstackgerrit | Verification of a change to openstack/ironic failed: Use trailing slash in the agent command URL https://review.opendev.org/720150 | 15:27 |
| jiriprox | Setting of is_smartnic in port connected to baremetal node seems to be not enough. I'm looking for someone who has experience with smartnic case. | 15:28 |
| jiriprox | Kernel driver is available as well. | 15:29 |
| *** Goneri has quit IRC | 15:30 | |
| jiriprox | It is using virtio driver. | 15:30 |
| *** Goneri has joined #openstack-ironic | 15:33 | |
| *** Qianbiao has quit IRC | 15:40 | |
| *** iurygregory_afk is now known as iurygregory | 15:41 | |
| TheJulia | jiriprox: Nothing in terms of IPA, but the smartnic integration flag is largely controlling how/when the machine is powered up because network programming via neutron needs the card powered up | 15:47 |
| *** ianychoi has joined #openstack-ironic | 15:47 | |
| *** jcoufal has quit IRC | 15:48 | |
| *** jcoufal has joined #openstack-ironic | 15:52 | |
| *** tkajinam has quit IRC | 15:53 | |
| jiriprox | TheJulia: Yes, that I've seen in https://review.opendev.org/#/c/583573/. Nevertheless I can't find anything what should be done on SmartNIC itself. If I understood it well then there is no procedure available for automatic OS deploy and configuration of SmartNIC. | 15:57 |
| patchbot | patch 583573 - ironic - Add support for Smart NICs (MERGED) - 34 patch sets | 15:57 |
| *** bdodd has quit IRC | 15:58 | |
| *** bdodd has joined #openstack-ironic | 16:01 | |
| *** lucasagomes has quit IRC | 16:09 | |
| *** dtantsur|brb is now known as dtantsur | 16:11 | |
| *** priteau has quit IRC | 16:12 | |
| *** jcoufal has quit IRC | 16:12 | |
| dtantsur | oh donnyd has dropped a bomb :) (re the ironic ML thread) | 16:13 |
| donnyd | please don't hurt me | 16:13 |
| arne_wiebalck | donnyd: ++ | 16:13 |
| *** ociuhandu has quit IRC | 16:20 | |
| *** jcoufal has joined #openstack-ironic | 16:20 | |
| donnyd | I feel like people may be about to send me 30 page responses... the lack of short ones at this point are semi concerning | 16:23 |
| dtantsur | :D | 16:25 |
| donnyd | this is also probably going to get me in trouble... but why is metal3 (kube wrapper for ironic) not using the opendev tooling? It would make life much mo easier wouldn't it ? | 16:27 |
| dtantsur | donnyd: I actually suggested it not so long ago. The response it that metal3 shares more development with k8s than with openstack. | 16:28 |
| dtantsur | ironic is largely consumed as is | 16:28 |
| donnyd | ??? LOL - its largely ironic with some kube tooling.. | 16:29 |
| donnyd | its clearly outside my scope, purpose and level of clout... just doesn't make a lot of sense is all | 16:31 |
| *** yaawang has joined #openstack-ironic | 16:32 | |
| donnyd | thanks for answer though dtantsur | 16:32 |
| dtantsur | donnyd: well, images with ironic is only a small part of the metal3 project. it's bigger part is the newly developed baremetal operator and some installer bits | 16:33 |
| arne_wiebalck | bye everyone o/ | 16:33 |
| dtantsur | see you arne_wiebalck | 16:33 |
| rpittau | bye, good night! o/ | 16:33 |
| donnyd | later arne_wiebalck | 16:33 |
| *** rpittau is now known as rpittau|afk | 16:33 | |
| *** yaawang_ has quit IRC | 16:34 | |
| donnyd | dtantsur: but functionally metal3 is a baremetal deployer for kube no ? | 16:34 |
| *** ociuhandu has joined #openstack-ironic | 16:34 | |
| donnyd | like I get the what is in development.. but what actual functions outside of native ironic stuff does it actually do ? ... I ask because I am curious | 16:35 |
| dtantsur | donnyd: it provides custom resources for bare metal provisioning and for adding bare metal nodes to the cluster | 16:36 |
| dtantsur | it does not expose ironic itself to the users though | 16:37 |
| donnyd | so it's CRD wrappers for ironic ? | 16:37 |
| donnyd | like translate this kube crd api call to an ironic one ? | 16:38 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: A boot partition on a GPT disk should be considered an EFI partition https://review.opendev.org/720184 | 16:39 |
| dtantsur | donnyd: essentially yes | 16:39 |
| donnyd | so now it makes even more sense to have it in the same CI ecosystem as ironic IMO | 16:40 |
| donnyd | but that is my +1 thought.. which means literally nothing | 16:40 |
| donnyd | because wouldn't someone want to know for sure in some form of vetted test infra that the kube crd api call does in fact do the thing you wanted it to do... | 16:42 |
| donnyd | like we already have the test methods, process, and validation for metal in opendev... but then metal3 is tested somewhere else completely... comical IMO | 16:43 |
| *** jcoufal has quit IRC | 16:45 | |
| dtantsur | I think they care less about how precise we deploy bare metal, but more for end-to-end testing in the k8s context. I.e. that we can start with powered off machines and end up with a working cluster. | 16:47 |
| donnyd | I don't want you to feel like I am harassing you here dtantsur - I am just curious is all | 16:48 |
| dtantsur | no worries, I'm totally fine (but will have to leave in ~ 10 mins) | 16:49 |
| donnyd | so the working cluster is provisioned by ironic? | 16:49 |
| donnyd | at the bottom end | 16:49 |
| dtantsur | donnyd: also worth noting that the people here are not really the main driving force behind the whole metal3. People writing the bare metal operators don't all have any openstack/opendev exposure. | 16:50 |
| dtantsur | yes, this is correct | 16:50 |
| donnyd | yea that makes sense | 16:51 |
| donnyd | just a though is all.. like kinda falling in like with the desire to move ironic to a top level project would make a ton of sense if metal3 was coming along with it | 16:52 |
| donnyd | inline* | 16:52 |
| dtantsur | it does make sense to me, but I'm not in charge there :) | 16:52 |
| donnyd | thanks for helping me to understand dtantsur | 16:53 |
| donnyd | :) | 16:53 |
| dtantsur | you're welcome | 16:53 |
| dtantsur | now, folks, I'm slowly calling it a day | 16:53 |
| *** dtantsur is now known as dtantsur|afk | 16:53 | |
| dtantsur|afk | o/ | 16:53 |
| *** k_mouza has quit IRC | 16:55 | |
| *** jcoufal has joined #openstack-ironic | 16:58 | |
| *** ociuhandu has quit IRC | 17:01 | |
| *** ociuhandu has joined #openstack-ironic | 17:02 | |
| *** priteau has joined #openstack-ironic | 17:05 | |
| *** ociuhandu has quit IRC | 17:08 | |
| *** ociuhandu has joined #openstack-ironic | 17:09 | |
| *** ricolin has quit IRC | 17:10 | |
| *** ociuhandu has quit IRC | 17:14 | |
| *** derekh has quit IRC | 17:17 | |
| openstackgerrit | Harald Jensås proposed openstack/ironic-inspector master: Port physical network hook https://review.opendev.org/718594 | 17:20 |
| *** priteau has quit IRC | 17:26 | |
| *** yolanda has joined #openstack-ironic | 17:27 | |
| TheJulia | calling it a day sounds really nice | 17:35 |
| TheJulia | jiriprox: so the way it is supposed to work is out of band network configuration. Ironic doesn't deploy that agent on to the smartnic because it is something a running OS shouldn't be able to see and in working with the neutron folks it was determined because of the tie-ins and the needs/ability requirement to tie things in, that the networking had to inter-op with neutron for that networking config | 17:37 |
| TheJulia | donnyd: A lot of that separation is also product driven in terms of decrees of how things shall be done | 17:38 |
| TheJulia | donnyd: and they also want to spin up lots of memory intensive resources, so I'm kind of all for it staying out of OpenDev since we maintain a stable API, they should bea ble to leverage that. | 17:39 |
| TheJulia | But I'm also kind of crazy | 17:39 |
| donnyd | TheJulia: So metal3 needs different flavors of test node for opendev to be more useful. That is what I heard you say | 17:42 |
| TheJulia | AIUI, yeah | 17:44 |
| TheJulia | Granted, I'm on the periphery of that discussion so my understanding may very well be wrong | 17:45 |
| *** ociuhandu has joined #openstack-ironic | 17:52 | |
| *** dsneddon has joined #openstack-ironic | 18:11 | |
| *** ociuhandu has quit IRC | 18:15 | |
| *** dsneddon has quit IRC | 18:16 | |
| *** ociuhandu has joined #openstack-ironic | 18:22 | |
| *** Lucas_Gray has joined #openstack-ironic | 18:23 | |
| *** jamesdenton_wfh has quit IRC | 18:27 | |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Add `network_data` field to ironic node object https://review.opendev.org/687910 | 18:30 |
| openstackgerrit | Christopher Dearborn proposed openstack/ironic master: DRAC: Added redfish management interface issue https://review.opendev.org/720076 | 18:31 |
| *** alexmcleod has quit IRC | 18:31 | |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Add `get_node_network_data` method to NetworkInterface https://review.opendev.org/688184 | 18:32 |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Add `get_node_network_data` to non-Neutron NetworkInterface https://review.opendev.org/688266 | 18:32 |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Make redfish boot interface to use `network_data` https://review.opendev.org/688272 | 18:32 |
| *** ociuhandu has quit IRC | 18:38 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Change [deploy]/default_boot_option to local https://review.opendev.org/492114 | 18:46 |
| TheJulia | Want to trade: Cute cat photos for reviews on: https://review.opendev.org/#/c/714212/ | 18:47 |
| patchbot | patch 714212 - ironic - "dual stack" support for PXE/iPXE - 11 patch sets | 18:47 |
| *** jcoufal has quit IRC | 18:49 | |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Make redfish boot interface to use `network_data` https://review.opendev.org/688272 | 18:50 |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Add `get_node_network_data` to Neutron NetworkInterface https://review.opendev.org/688896 | 18:52 |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Add L3 boot section to the docs https://review.opendev.org/689844 | 18:52 |
| * iurygregory agrees with TheJulia | 18:56 | |
| iurygregory | but I want cute panda photos XD | 18:56 |
| etingof | that patch is worth the whole litter of cute kittens | 18:58 |
| TheJulia | https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.pinterest.com%2Fpin%2F26880929001560383%2F&psig=AOvVaw3BTeabTxlh0eH4CtAk6V35&ust=1587064012947000&source=images&cd=vfe&ved=0CAIQjRxqFwoTCJC49aaQ6-gCFQAAAAAdAAAAABAD | 19:07 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-inspector master: Convert jobs to dib https://review.opendev.org/712516 | 19:13 |
| *** ociuhandu has joined #openstack-ironic | 19:23 | |
| jiriprox | TheJulia: Does the agent mean neutron-openvswitch-agent and ovs which should be installed on SmartNIC in advance? Is there any configuration guide available for SmartNC case? | 19:25 |
| TheJulia | jiriprox: as I understand it, yes. My understanding is it would be the same case of use as if your using a smartnic offloading with the host as a compute node. | 19:26 |
| * TheJulia wonders where that doc ended up | 19:27 | |
| TheJulia | https://specs.openstack.org/openstack/ironic-specs/specs/approved/support-smart-nic.html was the spec | 19:28 |
| TheJulia | for our side | 19:28 |
| *** ociuhandu has quit IRC | 19:29 | |
| iurygregory | would be worth asking in #openstack-neutron to confirm with them jiriprox | 19:31 |
| TheJulia | Yeah, I think it woudl be, I'm struggling to find docs and I thought there was stuff written up by Hamdyk | 19:35 |
| jiriprox | TheJulia: I've seen that guide.What is missing there and what I'm looking for is guide how to prepare SmartNIC side to be able to communicate with Ironic. | 19:36 |
| TheJulia | for a baremetal tenant machine? | 19:37 |
| jiriprox | Yes | 19:38 |
| TheJulia | https://specs.openstack.org/openstack/neutron-specs/specs/stein/neutron-ovs-agent-support-baremetal-with-smart-nic.html is the neutron spec | 19:38 |
| * TheJulia sighs | 19:40 | |
| TheJulia | https://review.opendev.org/#/c/586252/ was the patch for neutron | 19:40 |
| patchbot | patch 586252 - neutron - Add Smart NIC representor port to integration bridge (MERGED) - 46 patch sets | 19:40 |
| TheJulia | so it does look like hamdyk is still active | 19:41 |
| TheJulia | I feel like we should seek his input because as I understand it, the agents should have been pre-setup pointed at Neutron from with-in the smartnics and Ironic itself just uses the neutron REST api | 19:42 |
| *** yolanda has quit IRC | 19:50 | |
| jiriprox | TheJulia: This is my understanding as well. | 19:56 |
| *** yolanda has joined #openstack-ironic | 20:03 | |
| *** ociuhandu has joined #openstack-ironic | 20:04 | |
| TheJulia | I need to lay down for a little bit, I'm getting a migraine forming | 20:07 |
| TheJulia | jiriprox: do you want to reach out to hamdyk? Or do you want me to? I am guessing it will have to be an email | 20:12 |
| *** ociuhandu has quit IRC | 20:14 | |
| openstackgerrit | Harald Jensås proposed openstack/ironic-inspector master: Port physical network CIDR map hook https://review.opendev.org/718594 | 20:25 |
| openstackgerrit | Merged openstack/ironic master: Use trailing slash in the agent command URL https://review.opendev.org/720150 | 20:27 |
| *** jiriprox has quit IRC | 20:28 | |
| *** k_mouza has joined #openstack-ironic | 20:56 | |
| *** k_mouza has quit IRC | 21:00 | |
| *** ociuhandu has joined #openstack-ironic | 21:02 | |
| *** ociuhandu has quit IRC | 21:09 | |
| *** Lucas_Gray has quit IRC | 21:18 | |
| *** Lucas_Gray has joined #openstack-ironic | 21:24 | |
| *** Lucas_Gray has quit IRC | 21:29 | |
| *** jtomasek has quit IRC | 21:42 | |
| *** ociuhandu has joined #openstack-ironic | 21:57 | |
| *** Lucas_Gray has joined #openstack-ironic | 22:00 | |
| *** ociuhandu has quit IRC | 22:07 | |
| *** ociuhandu has joined #openstack-ironic | 22:08 | |
| *** ociuhandu has quit IRC | 22:13 | |
| * TheJulia sends email to mellenox contacts regarding smartnic docs. | 22:15 | |
| TheJulia | and mailbox not found for hamdyk | 22:16 |
| TheJulia | :( | 22:16 |
| TheJulia | Well, I know what happened now | 22:16 |
| *** jiriprox has joined #openstack-ironic | 22:23 | |
| * etingof is not happy that centos8 ramdisk fails to mount root | 22:42 | |
| *** rcernin has joined #openstack-ironic | 22:45 | |
| *** rloo has quit IRC | 22:45 | |
| *** Goneri has quit IRC | 22:47 | |
| *** tkajinam has joined #openstack-ironic | 22:59 | |
| * TheJulia blinks | 23:02 | |
| TheJulia | that seems like a major problem | 23:02 |
| TheJulia | donnyd: did you really expect people to throw blunt objects? This is ironic! We throw soft cuddly things... like... stuffed animals! | 23:06 |
| *** rh-jelabarre has quit IRC | 23:12 | |
| *** Lucas_Gray has quit IRC | 23:23 | |
| donnyd | I figured someone would at the least get a kick out of it | 23:32 |
| *** jiriprox has quit IRC | 23:33 | |
| *** ociuhandu has joined #openstack-ironic | 23:37 | |
| *** ociuhandu has quit IRC | 23:49 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!