| prometheanfire | should https://github.com/openstack/ironic/blob/master/ironic/common/glance_service/image_service.py#L299 be hard coded to switft (should it not allow radosgw?) | 03:08 |
|---|---|---|
| rpittau | good morning ironic! o/ | 08:31 |
| dtantsur | kubajj: "this" = my inspector spec? yes, absolutely. it's all published under a free license. | 08:56 |
| kubajj | dtantsur: yes, the spec. Thanks. I've skimmed through it yesterday and it clarifies some parts | 09:15 |
| dtantsur | prometheanfire: pardon my ignorance, is radosgw using a different service type? I thought it was more of a drop-in replacement. | 09:19 |
| opendevreview | Derek Higgins proposed openstack/sushy master: Add TransferProtocolType for any general error that mentions it https://review.opendev.org/c/openstack/sushy/+/878208 | 11:24 |
| iurygregory | morning Ironic | 12:23 |
| prometheanfire | dtantsur: object-storage is the service-type, the service-name is whatever we want to call it (variable), when creating the endpoint you reference the service name | 13:00 |
| prometheanfire | dtantsur: though that doesn't seem to be my problem, the nova ironic driver is throwing a 'An auth plugin is required to determine endpoint URL' | 13:03 |
| dtantsur | prometheanfire: oh, you're right, we should be using a service type | 13:04 |
| dtantsur | the 2nd error is probably misconfiguration | 13:04 |
| dtantsur | rpittau: our of curiosity, have you tried proposing a patch that breaks the metal3 job? or are we waiting for the source support in metal3-dev-env for that? | 13:06 |
| prometheanfire | ya, not sure where, since I'm using OSA, that error would be coming from nova-compute running in the ironic-compute container (if you are familiar)? | 13:06 |
| prometheanfire | and would be the nova config? | 13:06 |
| dtantsur | prometheanfire: "is coming from nova-compute" may mean the error in nova.conf (the [ironic] section), but it's possible that it is just proxying an error from ironic. | 13:07 |
| dtantsur | I guess you can check ironic logs for the latter. | 13:07 |
| prometheanfire | the traceback shows the ironic driver within nova at least | 13:10 |
| rpittau | dtantsur: metal3-dev-env main branch was actually broken for ironic until the variable change in https://github.com/metal3-io/metal3-dev-env/pull/1182 , or you mean something different? | 13:40 |
| rpittau | ah nvm | 13:40 |
| rpittau | you mean in ironic itself I guess | 13:40 |
| rpittau | we need the source support first, yeah | 13:41 |
| dtantsur | yep. somehow emulate a breaking change | 13:41 |
| dtantsur | okie | 13:41 |
| opendevreview | Verification of a change to openstack/ironic master failed: Use main branch of metal3-dev-env to run metal3 integration job https://review.opendev.org/c/openstack/ironic/+/877600 | 13:42 |
| rpittau | lol | 13:42 |
| opendevreview | Verification of a change to openstack/ironic master failed: Use main branch of metal3-dev-env to run metal3 integration job https://review.opendev.org/c/openstack/ironic/+/877600 | 13:51 |
| rpittau | dta | 13:55 |
| rpittau | mgreee | 13:55 |
| rpittau | dtantsur: I had 2 spare minutes https://review.opendev.org/c/openstack/project-config/+/878223 | 13:55 |
| dtantsur | Thanks! Maybe we'll need to add ironic-image eventually, but that can wait for sure. | 13:56 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-specs master: Merge Inspector into Ironic https://review.opendev.org/c/openstack/ironic-specs/+/878001 | 14:07 |
| dtantsur | Okay, this is ready for review | 14:07 |
| JayF | Ironic folks: please DM me an email address if you want meeting invites for Ironic vPTG events. I will be creating one for me personally to ensure I don't miss anything, and I'm happy to add people to the meeting invite. | 14:47 |
| dtantsur | PM'ed; also I've just advertised our PTG at the Metal3 weekly meeting! | 14:48 |
| JayF | dtantsur: see slack lol | 14:48 |
| JayF | dtantsur: I plugged it in slack literally 5 minutes ago, too | 14:48 |
| JayF | we are singing a duet | 14:48 |
| dtantsur | haha, great! | 14:48 |
| JayF | we will pull the k8s and openstack communities together through friendliness, force of will, and hardware provisioning | 14:49 |
| dtantsur | yep! we also had a quick look at the current schedule to spot topics of particular interest for metal3 people. | 14:49 |
| JayF | Operator Hour! | 14:50 |
| JayF | I want to see k8s metal3 operators at Ironic operator hour | 14:50 |
| JayF | that would really really be thrilling | 14:50 |
| dtantsur | Sure, but also firmware upgrades, IPA communication, cross-conductor communication, inspector merger | 14:50 |
| JayF | It's unclear to me how much the general metal3 community really cares about the implementation details of ironic vs just the APIs we present | 14:51 |
| dtantsur | well, these topics are details that matter | 14:51 |
| dtantsur | IPA communication is related to the TLS 1.3 questio | 14:51 |
| dtantsur | cross-conductor communication - to potential multi-conductor metal3 | 14:52 |
| dtantsur | firmware upgrades is a common request (not of interest for RH as well) | 14:52 |
| JayF | Yeah it's interesting, nowhere I've worked has really cared about {bios,raid,firmware}_interface | 14:53 |
| JayF | we always implemented it all in-band using generic steps in cleaning/deploy (including places like here, where they already had that in place without my influence) | 14:53 |
| dtantsur | daaamnn, I forgot auto-discovery in my spec.. | 14:59 |
| * dtantsur screaming inside | 14:59 | |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-specs master: [WIP] Migrate inspection rules from Inspector https://review.opendev.org/c/openstack/ironic-specs/+/878230 | 15:01 |
| JayF | have we considered class IronicInspector(object): def __init__(): subprocess.call('ironic-inspector') BOOM SERVICES INTEGRATED /s | 15:02 |
| JayF | it's seriously hard to ensure you get all the edge case features of something like that :/ | 15:02 |
| dtantsur | that will not solve the maintenance problems | 15:04 |
| dtantsur | not the performance issues of synchronizing ironic and inspector databases | 15:04 |
| JayF | Oh, it's 100% a tongue-in-cheek joke | 15:04 |
| dtantsur | I mean, yeah, it's a great idea :) | 15:04 |
| JayF | I am very stoked about the inspector integration work, with both upstream and downstream hats on | 15:05 |
| JayF | not often in OpenStack we can be like "here's a quantifyable decrease in complexity" | 15:05 |
| * dtantsur is probably too serious all of the sudden because he does not feel well | 15:05 | |
| dtantsur | decrease in complexity?? IN MY OPENSTACK??? | 15:05 |
| TheJulia | dtantsur: then get some rest() | 15:05 |
| TheJulia | My sanity + not over doing it is why I'm taking the next two days off | 15:06 |
| dtantsur | need to enlighten a person downstream now. then maybe do some sports or other non-computer activities | 15:06 |
| dtantsur | a good idea. my break is in just 2 (?) weeks | 15:06 |
| JayF | I actually have a question for you while you're here, this is a pretty good audience for it | 15:06 |
| JayF | Easter holiday observed is Apr 10 | 15:06 |
| TheJulia | .... in large part because I have an actual electrical fault to fix | 15:06 |
| JayF | I'm going to be gone that entire week (4/10 - 4/17) | 15:06 |
| JayF | Does someone else want to run the meeting or should we cancel it? | 15:06 |
| * TheJulia looks at calendars | 15:07 | |
| dtantsur | I'm out Apr 6 - Apr 11 | 15:07 |
| JayF | I have no real insight into how observed Easter Monday is internationally; it's waning in observation in the US... | 15:07 |
| dtantsur | Thu to Tue | 15:07 |
| rpittau | JayF: I think we should cancel it, I'm also out | 15:07 |
| dtantsur | (both Friday and Monday are public holidays in Germany) | 15:07 |
| JayF | I think that's pretty close to quorum for people who participate in meetings | 15:07 |
| JayF | so I'm going to cancel it :) | 15:07 |
| TheJulia | I'll be aroudn that day, but it sounds like we just won't have critical mass | 15:08 |
| JayF | or you should take it as a cosmic sign to take the day off :) | 15:08 |
| TheJulia | ... true | 15:08 |
| TheJulia | I'll figure that out the week before, I have Jury duty that week | 15:09 |
| TheJulia | so there is just a huge unknown for me | 15:09 |
| TheJulia | so I may, or may not be around the week of the 3rd, I just don't know. | 15:10 |
| TheJulia | California has some funny rules and expectations which only really work if your employed by a CA based company or retired. | 15:13 |
| dtantsur | Countries inside of countries :) | 15:13 |
| TheJulia | ... legitimate. | 15:13 |
| dtantsur | I'm not sure about employment rules, but public holidays differ in different German states. And since I don't live in the state I'm employed it, it is sometimes confusing :) | 15:14 |
| TheJulia | Yeah, same basic issue exists here, but generally employers only honor the federal level holidays | 15:14 |
| jssfr | that's not a choice here, at least :D | 15:15 |
| jssfr | (here := germany) | 15:15 |
| rpittau | in France some regions have different bank holidays, like Alsatia | 15:15 |
| dtantsur | Since I'm employed in Bavaria, I tend to prefer its holidays - they have the most of them :D | 15:15 |
| dtantsur | rpittau: interesting, I thought France is pretty uniform in terms of regulations | 15:16 |
| rpittau | Alsace actually, Alsatia is Latin :P | 15:16 |
| dtantsur | mmm, a man of education you are, I see, I see | 15:17 |
| rpittau | :D | 15:17 |
| TheJulia | heh | 15:17 |
| dtantsur | "OpenStack 2023.1 Antelope is officially released!" congrats folks \o/ | 15:19 |
| JayF | That's a lot of antelopes. | 15:20 |
| JayF | I hope I don't run into that 1/10th of an Antelope. Sounds horrible. | 15:20 |
| rpittau | \o/ | 15:20 |
| rpittau | "release the Antelope" | 15:21 |
| prometheanfire | dtantsur: was able to get a backtrace into a paste (getting logs is a fun process from this env) https://paste.openstack.org/show/819263/ | 15:22 |
| dtantsur | prometheanfire: this is coming from somewhere inside ironic, I suspect | 15:22 |
| dtantsur | ironic/conductor/deployments.py to be even more specific | 15:23 |
| prometheanfire | ok, will look at the conductor logs more closely | 15:23 |
| prometheanfire | dtantsur: is the logs from the contuctor https://paste.openstack.org/show/819264/ | 15:25 |
| prometheanfire | I did duplicate the radosgw service name'd endpoint to a swift named endpoint just in case too | 15:26 |
| dtantsur | prometheanfire: the [swift] section is misconfigured, it's one step before the catalog is even accessed | 15:27 |
| prometheanfire | ok, ya that section is empty, time to look at why that is in OSA | 15:27 |
| prometheanfire | hmm, it's always empty... | 15:28 |
| prometheanfire | you sure it's not the glance section? | 15:29 |
| TheJulia | I'm thinking the swift section as well | 15:37 |
| TheJulia | well... | 15:37 |
| TheJulia | I *believe* tempurls get generated without talking to swift though | 15:38 |
| JayF | You need the external url to swift to generate those urls, yeah? | 15:38 |
| JayF | And the err isn't connecting to swift; it's about finding the endpoint yeah? | 15:39 |
| TheJulia | https://github.com/openstack/ironic/blob/master/ironic/common/glance_service/image_service.py#L299 | 15:39 |
| TheJulia | indeed | 15:39 |
| JayF | bingo :D I thoguht so | 15:39 |
| JayF | I actually remember that code lol | 15:39 |
| JayF | part of the giant terrible agent driver PR of 2014 lol | 15:39 |
| prometheanfire | lol | 15:40 |
| TheJulia | heh | 15:40 |
| prometheanfire | hence why I linked to that about hardcoding swift there yesterday | 15:40 |
| dtantsur | I actually suspect we misinterpreted that | 15:41 |
| dtantsur | it's not hardcoding a name from the catalog, it's just referring to [swift] in ironic.conf | 15:41 |
| * TheJulia ponders for a minute | 15:41 | |
| prometheanfire | oh | 15:41 |
| TheJulia | dtantsur++ | 15:41 |
| prometheanfire | now it's making more sense | 15:41 |
| TheJulia | the underlying issue is it can't figure out the endpoint because you *can* have ceph registered as swift in the service catalog | 15:41 |
| TheJulia | or at least, as dmitry put it, reference in the config | 15:42 |
| prometheanfire | the swift section of my config is empty, let me fix that first :P | 15:42 |
| TheJulia | once the swift client is launched, then we generate the tempurl, we don't actually talk to swift for that if memory serves | 15:42 |
| dtantsur | yeah, I don't think it involves talking to swift | 15:43 |
| TheJulia | Yeah, it just needs to know the URL for the api service to generate the url | 15:45 |
| rpittau | good night! o/ | 16:53 |
| opendevreview | Verification of a change to openstack/ironic master failed: Use main branch of metal3-dev-env to run metal3 integration job https://review.opendev.org/c/openstack/ironic/+/877600 | 17:16 |
| JayF | rpittau: TheJulia: for purposes of the OpenInfra live tomorrw, what's a ballpark on how much we improved port/portgroup listing last cycle? | 18:47 |
| TheJulia | hjensas: ^ | 18:47 |
| TheJulia | heh, the benchmark is broken | 18:49 |
| TheJulia | again | 18:49 |
| TheJulia | 20k nodes from the database in 1.2 seconds, not bad | 18:49 |
| opendevreview | Julia Kreger proposed openstack/ironic master: [CI] Fix port list benchmark https://review.opendev.org/c/openstack/ironic/+/878245 | 18:52 |
| TheJulia | I think we were averaging a couple hundred a second previously, ^ should tell us | 18:52 |
| JayF | 20k in 1.2 vs 200 in 1.2 is a 100x increase. That's stunning. | 18:54 |
| TheJulia | no | 18:54 |
| TheJulia | just from the db level | 18:54 |
| TheJulia | that excludes object | 18:54 |
| TheJulia | the couple hundred was post object api code returning | 18:55 |
| TheJulia | that test was broken by the shards support | 18:55 |
| JayF | aha | 18:55 |
| TheJulia | s/test/benchmark/ | 18:55 |
| JayF | that makes more sense | 18:55 |
| JayF | wait, what? how? | 18:55 |
| JayF | oh, that makes sense heh | 18:55 |
| TheJulia | insertion of a required argument in the middle of hte list | 18:55 |
| JayF | benchmarks use internal APIs, that makes sense | 18:56 |
| TheJulia | its a private method | 18:56 |
| TheJulia | yup | 18:56 |
| JayF | you had me worried I API-broke a thing lol | 18:56 |
| TheJulia | oh, sorry | 18:56 |
| TheJulia | so we should have an idea in a little bit | 18:56 |
| JayF | OK; I have until EOD for these slides. I didn't put it off until the last minute <.< >.> | 18:56 |
| JayF | You think it's safe to say Ironic RBAC support is done, with service user added? | 18:57 |
| JayF | s/user/role/ | 18:57 |
| TheJulia | we don't have a bunch of testing in our tempest plugin, but that might be okay until we remove the legacy policy support at some point | 18:58 |
| TheJulia | I think we have the needful "use this credential" logic | 18:58 |
| JayF | I'll say we're done with feature support for it | 18:58 |
| JayF | we're always improving testing :) | 18:58 |
| TheJulia | yeah, unless something else drops that we find we need to do, I feel done with it | 18:59 |
| TheJulia | ... which is also why I feel good about giving a talk on the subject | 18:59 |
| TheJulia | :) | 18:59 |
| opendevreview | Verification of a change to openstack/ironic master failed: Enables boot modes switching with Anaconda deploy for ilo driver https://review.opendev.org/c/openstack/ironic/+/860821 | 19:05 |
| JayF | TheJulia: RFR slides 18-22 here https://docs.google.com/presentation/d/1KFZ2qzkAyJYG3qiVeuy4vKd1NFQzK6tkHBX2929_OFg/edit#slide=id.g222764c32b6_4_22 | 19:32 |
| JayF | other than the clearly marked TODO, anything I missed or am misstating? | 19:32 |
| * TheJulia suspects her intertubes are full of sleeping lolcats | 19:34 | |
| TheJulia | grrrrrr | 19:37 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM change to trigger benchmark job https://review.opendev.org/c/openstack/ironic/+/878249 | 19:38 |
| TheJulia | and finally running | 19:43 |
| TheJulia | JayF: is the commit count just ironic the repo itself, or everything under ironic's governance | 19:46 |
| JayF | everything under ironic governance | 19:46 |
| JayF | according to stackalytics | 19:46 |
| JayF | I'll make a note to mention that specifically when presenting | 19:47 |
| TheJulia | k | 19:47 |
| JayF | 23 packages, wow | 19:47 |
| * TheJulia watches bifrost spin on the benchmark job | 19:48 | |
| * JayF adds the number of packages | 19:48 | |
| TheJulia | "Ironic is not a small project, it is a very BIG project" | 19:50 |
| JayF | that's the idea I try to get across to almost everyone I talk to | 19:50 |
| JayF | you use metal3? Thanks for using Ironic. | 19:50 |
| JayF | You run $newShinyBMTool? Let me tell you about the awesome CI tools we maintain to help | 19:50 |
| TheJulia | can we have an AI generate us a strongbad email style video talking about how ironic is a very BIG project!? | 19:50 |
| JayF | be careful what you wish for | 19:51 |
| * TheJulia waits with anticipation | 19:51 | |
| JayF | > /imagine pixie boots, the openstack ironic mascot in a lucha style wrestling match with Strong Bad from Homestarrunner.com | 19:52 |
| JayF | this is not going to work, but I had to try. Don't let me down MidJourney! | 19:52 |
| * TheJulia is curious what it comes back with | 19:52 | |
| JayF | 4 cartoon scenes that look like something from space ghost coast to coast | 19:53 |
| JayF | https://cdn.discordapp.com/attachments/306957680734371840/1088188632726241421/grid_0.webp | 19:53 |
| TheJulia | oh my... | 19:54 |
| JayF | it's easy to create whimsy, it's hard to create something specific and silly | 19:54 |
| JayF | TheJulia: so I take lack of further comment/feedback as a +1 to those update slides? | 19:59 |
| TheJulia | JayF: yeah, no other comments | 20:09 |
| TheJulia | JayF: well, aside from the mascot one | 20:10 |
| JayF | TheJulia: ...where can I get that image? | 20:10 |
| JayF | I picked the one I did because it's on our wiki page, ez to find and use | 20:11 |
| TheJulia | https://www.openstack.org/project-mascots/ | 20:11 |
| JayF | https://www.openstack.org/project-mascots/ | 20:11 |
| JayF | TheJulia: added a small grey drop shadow so the drumsticks pop too | 20:13 |
| TheJulia | cool cool | 20:13 |
| TheJulia | That works | 20:13 |
| TheJulia | JayF: Took 7.773931264877319 seconds to return all 20000 ports via ports API call pattern | 20:20 |
| zigo | FYI, ironic-python-agent just entered Debian Unstable. | 20:21 |
| TheJulia | zigo: anything we need to be on the eye out for? | 20:21 |
| TheJulia | or remedy? | 20:21 |
| zigo | I was wondering: is it ok to just boot a debian-live system over PXE with that package installed, instead of the kernel + ramdisk made with dib ? | 20:22 |
| zigo | I've put all what I saw in the dib elements directly on that package... :P | 20:22 |
| TheJulia | i mean... if the debian-live system knows to start the ramdisk, I wouldn't see why not | 20:22 |
| TheJulia | we do expect dhcp address acqusition to occur outside of the agent running though | 20:23 |
| zigo | TheJulia: What I did with https://salsa.debian.org/openstack-team/debian/openstack-cluster-installer was using a custom debian-live image built with live-build. | 20:24 |
| zigo | It's the standard Debian kernel + ramdisk, with a script in the ramdisk that fetches a squashfs over HTTP, which is much nicer and faster than a 400 GB ramdisk... | 20:24 |
| zigo | I'm planning to do the same with Ironic. | 20:25 |
| JayF | That is similar-ish to things I've done in the past downstream with IPA; but I will say that would not work quite as well these days | 20:25 |
| TheJulia | 400MB, not GB right? | 20:25 |
| JayF | operators can choose different ramdisks to boot per server | 20:26 |
| zigo | TheJulia: I'm currently writing https://salsa.debian.org/openstack-team/debian/supply | 20:26 |
| zigo | Yeah 400 MB, sorry. | 20:26 |
| JayF | so I'd wonder how you'd communicate the info through -- just hardcode squashfs urls into the ramdisk? | 20:26 |
| JayF | but there's no OS requirement; just that IPA is running, the node is up with an address, and that's it | 20:26 |
| zigo | JayF: live-build use the same technique as IPA, it reads the kernel command line. | 20:26 |
| JayF | IPA doesn't use kernel command line to dictate what image | 20:26 |
| zigo | That's how it knows where's the squashfs to wget. | 20:27 |
| JayF | the image is the image | 20:27 |
| TheJulia | zigo: ahh, okay. That makes sense and sort of as JayF was saying, I've heard people do. Pretty much anything to avoid tftping large amounts of data which is why many prefer ipxe | 20:27 |
| JayF | IPA just uses kernel command line for some configuration items | 20:27 |
| JayF | zigo: one thing is: do not assume actual-internet access from an IPA ramdisk running on a server. | 20:27 |
| zigo | Yeah, tftp / udp is bad... | 20:27 |
| zigo | JayF: My OpenStack installer also works completely offline. | 20:28 |
| zigo | So no worries. | 20:28 |
| JayF | zigo: also; traditionally I was extremely -1 to ipa being packaged for general `apt-get install ironic-python-agent` because it seems like a bad idea to let someone install, on their system, a daemon designed to erase it :) | 20:28 |
| zigo | The only thing you need is access to a Debian repo ... :P | 20:28 |
| JayF | So I'm extremely onboard to help you figure out + troubleshoot a debian-based image using debian methods | 20:28 |
| zigo | (we run a double-server mirror...) | 20:28 |
| JayF | I'm extremely skeptical about that package entering the general debian package repos | 20:28 |
| TheJulia | at least the agent won't do anything if it can't talk to an ironic | 20:29 |
| JayF | I guess we did remove --standalone which lowers the risk somewhat | 20:29 |
| zigo | JayF: As I told you, my IPA package just entered Debian today... :) | 20:29 |
| JayF | yeah that's why I'm :(-ing about it at you :D | 20:29 |
| zigo | So that's my 508'th OpenStack package in Debian... :P | 20:30 |
| TheJulia | JayF: the agent has to also lookup itself for that part of the service to even start the api which then requires a token now | 20:31 |
| opendevreview | Julia Kreger proposed openstack/ironic-specs master: WIP: cross-conductor rpc https://review.opendev.org/c/openstack/ironic-specs/+/873662 | 20:31 |
| JayF | TheJulia: yeah but when standalone existed, all those requirements were gone | 20:33 |
| TheJulia | yup | 20:33 |
| JayF | TheJulia: I forget we pulled it out (I think *I* pulled it out actually, making this double-sad) | 20:33 |
| TheJulia | would cookies help?! | 20:33 |
| JayF | probably reduction in my consumption of cookies would help lol | 20:36 |
| TheJulia | heh | 20:39 |
| * TheJulia suddenly wants cookies | 20:39 | |
| JayF | > Took 7.773931264877319 seconds to return all 20000 ports via ports API call pattern. | 20:39 |
| * TheJulia instead thinks checking on current benchy is needed | 20:40 | |
| JayF | from the CI job | 20:40 |
| TheJulia | yup | 20:40 |
| JayF | now I need to find an old one? | 20:40 |
| JayF | no, those logs will be gone | 20:40 |
| TheJulia | I was just going to say | 20:40 |
| JayF | do you remember roughtly results of old benchmark? | 20:40 |
| TheJulia | I think it was ~200, if you go back to the original change, I think hjensas left comments in the gerrit history | 20:40 |
| TheJulia | it might have been 250/second, but it was definitely not great | 20:41 |
| JayF | > Took 147.8800814151764 seconds to return all 20000 ports via ports API call pattern. | 20:42 |
| JayF | WOOF | 20:42 |
| JayF | https://paste.opendev.org/show/boW6bFAIxuh4bMcYsvse/ | 20:42 |
| JayF | so a 20x improvement in port list API returns is probably safe to claim? | 20:42 |
| JayF | that's very safe, based on that benchmark from just the one change | 20:43 |
| TheJulia | yeah | 20:43 |
| opendevreview | Merged openstack/ironic master: [CI] Fix port list benchmark https://review.opendev.org/c/openstack/ironic/+/878245 | 21:12 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
