| *** altlogbot_1 has quit IRC | 00:23 | |
| *** irclogbot_0 has quit IRC | 00:23 | |
| *** mattw4 has quit IRC | 00:29 | |
| *** threestrands has quit IRC | 00:32 | |
| *** jamesmcarthur has joined #zuul | 01:02 | |
| *** jamesmcarthur has quit IRC | 01:12 | |
| *** jamesmcarthur has joined #zuul | 01:14 | |
| *** igordc has quit IRC | 01:16 | |
| *** jamesmcarthur has quit IRC | 01:24 | |
| *** irclogbot_0 has joined #zuul | 01:35 | |
| *** irclogbot_0 has quit IRC | 01:39 | |
| *** irclogbot_2 has joined #zuul | 01:42 | |
| *** irclogbot_2 has quit IRC | 01:43 | |
| *** bhavikdbavishi has joined #zuul | 02:17 | |
| *** bhavikdbavishi has quit IRC | 02:21 | |
| *** spsurya has joined #zuul | 02:44 | |
| *** irclogbot_2 has joined #zuul | 02:55 | |
| *** irclogbot_2 has quit IRC | 02:59 | |
| *** ianychoi has quit IRC | 03:12 | |
| *** ianychoi has joined #zuul | 03:13 | |
| *** bhavikdbavishi has joined #zuul | 03:19 | |
| *** jamesmcarthur has joined #zuul | 03:23 | |
| *** irclogbot_0 has joined #zuul | 03:35 | |
| *** irclogbot_0 has quit IRC | 03:39 | |
| *** jamesmcarthur has quit IRC | 03:54 | |
| *** jamesmcarthur has joined #zuul | 03:55 | |
| *** jamesmcarthur has quit IRC | 04:02 | |
| *** irclogbot_0 has joined #zuul | 04:03 | |
| *** jamesmcarthur has joined #zuul | 04:04 | |
| *** irclogbot_0 has quit IRC | 04:07 | |
| *** irclogbot_3 has joined #zuul | 04:23 | |
| *** irclogbot_3 has quit IRC | 04:33 | |
| *** jank has joined #zuul | 04:35 | |
| *** jamesmcarthur has quit IRC | 04:37 | |
| *** raukadah is now known as chandankumar | 04:55 | |
| *** bjackman has joined #zuul | 05:00 | |
| *** jamesmcarthur has joined #zuul | 05:04 | |
| *** bjackman has quit IRC | 05:20 | |
| *** irclogbot_0 has joined #zuul | 05:22 | |
| *** irclogbot_0 has quit IRC | 05:25 | |
| *** bjackman has joined #zuul | 05:25 | |
| *** bjackman_ has joined #zuul | 05:29 | |
| *** bjackman has quit IRC | 05:32 | |
| *** rfolco has quit IRC | 06:00 | |
| *** jamesmcarthur has quit IRC | 06:04 | |
| *** irclogbot_3 has joined #zuul | 06:18 | |
| *** irclogbot_3 has quit IRC | 06:21 | |
| *** irclogbot_1 has joined #zuul | 06:25 | |
| *** irclogbot_1 has quit IRC | 06:29 | |
| *** jank has quit IRC | 06:40 | |
| *** sshnaidm|pto is now known as sshnaidm|afk | 06:45 | |
| *** zbr has quit IRC | 06:58 | |
| *** zbr has joined #zuul | 07:08 | |
| *** toabctl has quit IRC | 07:11 | |
| *** nborg1 has joined #zuul | 07:14 | |
| *** toabctl has joined #zuul | 07:16 | |
| *** hashar has joined #zuul | 07:24 | |
| *** tosky has joined #zuul | 07:25 | |
| *** jamesmcarthur has joined #zuul | 07:29 | |
| *** irclogbot_2 has joined #zuul | 07:32 | |
| *** jamesmcarthur has quit IRC | 07:34 | |
| *** irclogbot_2 has quit IRC | 07:35 | |
| *** jpena|off is now known as jpena | 07:39 | |
| *** themroc has joined #zuul | 07:41 | |
| *** pcaruana has joined #zuul | 07:42 | |
| *** mhu has quit IRC | 07:45 | |
| *** irclogbot_1 has joined #zuul | 07:53 | |
| *** irclogbot_1 has quit IRC | 07:53 | |
| *** mkisielewski has joined #zuul | 08:01 | |
| *** mhu has joined #zuul | 08:11 | |
| *** sanjayu_ has joined #zuul | 08:23 | |
| *** flaper87 has joined #zuul | 08:47 | |
| flaper87 | I've set an autohold for one of my jobs but the VM keeps being deleted. Is there a way for me to debug why? The job name, tenant name, etc seem correct | 08:48 |
|---|---|---|
| ofosos | If I updated the gate job in a change, would the gate run with the updated job or would it run with the one one the branch I want to merge to? | 09:00 |
| AJaeger | in an untrusted repository: With the updated job. In a trusted repo: With the previous one. | 09:01 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - add support for git.tag.creation event https://review.opendev.org/679938 | 09:02 |
| nborg1 | When an untrusted project is retrying a job due to a failing pre-job, and the corresponding trusted configuration project has has a new commit merged, which version of the trusted configuration project is used in the retry? I assume the commit the job started with? | 09:09 |
| *** irclogbot_1 has joined #zuul | 09:10 | |
| *** irclogbot_1 has quit IRC | 09:11 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - fix wrong commit gitweb url https://review.opendev.org/679946 | 09:30 |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 09:41 | |
| *** hashar has quit IRC | 09:41 | |
| flaper87 | AJaeger: do you have any hint on my autohold question? | 09:43 |
| flaper87 | can't understand why the VM is not being held | 09:43 |
| flaper87 | it used to work, tho... I think | 09:43 |
| flaper87 | :D | 09:43 |
| flaper87 | ohhhh, I think I know | 09:43 |
| flaper87 | it's for failed jobs and this one is passing | 09:44 |
| *** bhavikdbavishi has quit IRC | 09:44 | |
| flaper87 | T_T | 09:44 |
| mkisielewski | Hi everyone. I've got some issues with reporter in pipeline using GitHub connection configured with GitHub App. It looks like repos are processed locally (merger, executor), but I can't see any reports (comments, status changes, merge) on Github. Debug logs don't show any errors. Where should I look to troubleshoot that? | 10:09 |
| *** badboy has joined #zuul | 10:20 | |
| *** mhu has quit IRC | 10:27 | |
| *** irclogbot_2 has joined #zuul | 10:30 | |
| *** irclogbot_2 has quit IRC | 10:33 | |
| *** badboy has quit IRC | 10:41 | |
| *** irclogbot_3 has joined #zuul | 10:44 | |
| *** irclogbot_3 has quit IRC | 10:45 | |
| *** bhavikdbavishi has joined #zuul | 11:03 | |
| *** bhavikdbavishi1 has joined #zuul | 11:06 | |
| *** bhavikdbavishi has quit IRC | 11:07 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 11:07 | |
| *** zbr is now known as zbr|swimming | 11:23 | |
| *** jpena is now known as jpena|lunch | 11:28 | |
| *** irclogbot_1 has joined #zuul | 11:44 | |
| tristanC | mkisielewski: does your pipelines have a github: {comment: true} reporter for success/failure? | 11:46 |
| *** irclogbot_1 has quit IRC | 11:48 | |
| *** electrofelix has joined #zuul | 11:55 | |
| *** mhu has joined #zuul | 12:00 | |
| *** bjackman_ has quit IRC | 12:01 | |
| *** bjackman_ has joined #zuul | 12:03 | |
| *** rfolco has joined #zuul | 12:08 | |
| *** hashar has joined #zuul | 12:09 | |
| *** jamesmcarthur has joined #zuul | 12:09 | |
| *** jamesmcarthur has quit IRC | 12:13 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 12:15 |
| *** jpena|lunch is now known as jpena | 12:32 | |
| *** zbr|swimming is now known as zbr | 12:42 | |
| *** hashar has quit IRC | 12:57 | |
| mkisielewski | tristanC: yes, exactly. I wonder if the Webhook url is correct, as in response (200) I get body containing - "You need to enable JavaScript to run this app." | 13:01 |
| pabelanger | never seen that before | 13:03 |
| pabelanger | you are seeing that in the github event? | 13:03 |
| mkisielewski | pabelanger: In GitHub App > Advanced - Recent deliveries - Response | 13:07 |
| pabelanger | interesting | 13:08 |
| pabelanger | I've not seen tht | 13:08 |
| pabelanger | so, maybe you are right the URL isn't correct | 13:08 |
| pabelanger | you can look in your web-debug.log and see if you can see the POSTs | 13:08 |
| mkisielewski | url should look like: http://<fqdn>/api/connection/<connection name>/payload ? Assuming that there's Apache Reverse Proxy in front. | 13:12 |
| pabelanger | yes, that looks correct: https://zuul-ci.org/docs/zuul/admin/drivers/github.html#application | 13:13 |
| pabelanger | you can then check apache logs | 13:13 |
| mkisielewski | I see POST request in apache, but not in web-debug.log. I'm using different config files per zuul service, so in this case should connection be defined also in web config or just scheduler, executor and merger? | 13:15 |
| pabelanger | mkisielewski: https://zuul-ci.org/docs/zuul/admin/components.html should show all the server itergation, that will help a little. | 13:17 |
| pabelanger | but yah, web service needs a few sections. like gearman, web, zookeeper, connections | 13:18 |
| pabelanger | https://github.com/ansible-network/windmill-config/blob/master/zuul/zuul.conf.j2 is an example of how we break them out for zuul.a.c | 13:18 |
| pabelanger | maybe not routing properly between apache and zuul-web | 13:19 |
| mkisielewski | pabelanger: Thanks, I've spent a lot of time digging through this documentation. I will give another try with connections described in web config. Seems like it should be it. | 13:20 |
| mkisielewski | Thanks for that example, it will really help. I regret that I didn't saw it before. | 13:22 |
| mkisielewski | pabelanger: Adding connection definition to web config file helped. Thanks a lot. | 13:33 |
| pabelanger | ++ | 13:34 |
| flaper87 | is there a job example that uses nodepool's k8s driver somewhere? Maybe in OpenStack or Ansible? I'd like to see how the job is defined and the pre/post tasks | 13:38 |
| Shrews | flaper87: we don't use that driver yet in those projects, but in theory, the job shouldn't care about the nodepool driver | 13:45 |
| flaper87 | Shrews: that's my assumption as well. Something is not quite right with my jobs, tho. Lemme see if I can narrow the issue | 13:47 |
| flaper87 | Shrews: basically, no tasks are being executed as if the ansible inventory was empty | 13:47 |
| corvus | flaper87: tristanC has experience using it and may be able to help | 13:49 |
| flaper87 | although, the tasks under a play that uses `- hosts: localhost` seem to be executed | 13:49 |
| flaper87 | corvus: awesome, thanks for directing me to tristanC | 13:49 |
| flaper87 | hopefully he's around and/or knows a bit more about this | 13:50 |
| Shrews | welcome back, corvus | 13:53 |
| corvus | Shrews: thanks! | 13:53 |
| flaper87 | so, I can confirm that `all` is not in the inventory_hostname when running the job on a pod | 13:57 |
| flaper87 | I guess I could force it by having a task that adds it | 13:57 |
| *** jamesmcarthur has joined #zuul | 13:58 | |
| *** jamesmcarthur has quit IRC | 14:03 | |
| *** jamesmcarthur has joined #zuul | 14:19 | |
| *** mkisielewski has quit IRC | 14:19 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 14:27 |
| tristanC | flaper87: here are our openshift based jobs: https://softwarefactory-project.io/cgit/config/tree/zuul.d/_jobs-openshift.yaml | 14:35 |
| tristanC | flaper87: in particular, here is the task to prepare the workspace: https://softwarefactory-project.io/cgit/config/tree/playbooks/openshift/unprivileged-machine.yaml | 14:35 |
| tristanC | Shrews: the job needs to care about the driver as the synchronize Ansible module doesn't work for kubectl connection | 14:36 |
| flaper87 | tristanC: I wish I could hug you right now | 14:37 |
| Shrews | tristanC: is that documented somewhere? | 14:37 |
| flaper87 | tristanC: what image are you using? I'd assume you have a pre-built image with some extra packages in it, right? | 14:38 |
| flaper87 | Do you have a dockerfile you can share? | 14:38 |
| tristanC | Shrews: in the spec, it does mentions "Not all jobs will be expected to work in containers. Some frequently used Ansible modules will not behave as expected when run with the kubectl connection plugin. Synchronize, in particular, may be problematic" | 14:39 |
| tristanC | flaper87: here is how software factory integration test validate openshift jobs: https://softwarefactory-project.io/cgit/software-factory/sf-ci/tree/roles/health-check/openshift/tasks | 14:40 |
| tristanC | flaper87: for base-openshift-native, we rely on BuildConfig to build the project image | 14:41 |
| *** jamesmcarthur has quit IRC | 14:43 | |
| tristanC | flaper87: otherwise we let project push images to a public registry and use them in nodepool pod label | 14:43 |
| jangutter | Q: Can you pass an ansible variable in a pre play to a post play within a job (without persisting it on disk or something)? | 14:44 |
| pabelanger | jangutter: no, needs to be on disk | 14:45 |
| jangutter | pabelanger: thanks! | 14:45 |
| dmsimard | pabelanger: zuul_return ? | 14:50 |
| flaper87 | tristanC: thanks a lot, I'll dig into this. Do you know how the `run` playbook is copied to the pod? | 14:52 |
| flaper87 | is it just mounted? | 14:52 |
| tristanC | flaper87: you're welcome, we also tried to share those job with upstream there: https://review.opendev.org/#/c/570669/, but the effort stalled | 14:52 |
| *** mattw4 has joined #zuul | 14:53 | |
| tristanC | flaper87: the `run` playbook is not copied, the openshift resources are regular nodepool object passed to the executor, the job playbooks are still running from the zuul-executor service | 14:53 |
| pabelanger | dmsimard: Oh, maybe. I keep forgetting about that | 14:54 |
| pabelanger | jangutter: listen to dmsimard | 14:54 |
| dmsimard | jangutter: there is a zuul_return module that might help carry things from one playbook (or job) to the next | 14:54 |
| pabelanger | https://zuul-ci.org/docs/zuul/user/jobs.html#return-values | 14:55 |
| jangutter | dmsimard: eenteresting - I know it transfers between jobs, but we're looking at something set in a pre play to be accessed in a post play. | 14:55 |
| tristanC | iirc, zuul_returned data are not passed between pre/run/post, it may only be used by child buildset jobs | 14:55 |
| dmsimard | tristanC: ah, thanks | 14:55 |
| pabelanger | yah, I think that is correct | 14:56 |
| jangutter | dmsimard: but thanks! | 14:56 |
| pabelanger | but, I haven't tested it in a while | 14:56 |
| dmsimard | jangutter: can otherwise try /etc/ansible/facts.d | 14:57 |
| corvus | jangutter: the workspace does persist, so you can write a file there and read it later | 14:57 |
| jangutter | yeah - it's no biggie, I was just wondering if there's a mechanism inside ansible. | 14:58 |
| pabelanger | dmsimard: we should add a role for that, I've always wanted to try that way | 14:59 |
| pabelanger | never have | 14:59 |
| flaper87 | tristanC: I guess I don't fully understand how one would run some commands in the container. Maybe `kubectl exec` or something like that | 14:59 |
| tristanC | flaper87: when using a pod label, then you have access to a kubectl connection and you can run command using regular ansible tasks | 15:02 |
| *** chandankumar is now known as raukadah | 15:02 | |
| tristanC | flaper87: when using a namespace label, then you first have to build and/or create a pod, which you can add_host to your inventory and proceed to use regular ansible tasks | 15:02 |
| tristanC | flaper87: you can find sample zuul.yaml and the relevant playbooks in this file: https://softwarefactory-project.io/cgit/software-factory/sf-ci/tree/roles/health-check/openshift/tasks/demo_project_zuul_configuration.yml | 15:04 |
| flaper87 | tristanC: so, when using a pod label I can just define my playbooks/tasks as usual `- hosts: all ...` The big difference is that I have to create a pod when using a namespace label | 15:06 |
| tristanC | flaper87: yes, that's correct | 15:07 |
| tristanC | flaper87: well, you don't have to create a pod with namespace label, you can just apply some files and assert the resources are created using lookups | 15:12 |
| flaper87 | right | 15:12 |
| *** jamesmcarthur has joined #zuul | 15:13 | |
| *** themroc has quit IRC | 15:16 | |
| clarkb | corvus: when you get a chance you may want to rereview https://review.opendev.org/#/c/679145/ and take a look at https://review.opendev.org/#/c/679819/ to continue on the swift improvements | 15:21 |
| clarkb | https://review.opendev.org/#/c/679814/ is an easy related docs update too | 15:21 |
| corvus | clarkb: you did not like my idea to make the name neutral? | 15:23 |
| corvus | like, there's nothing "object storage" about it. it's just "shard better" | 15:24 |
| clarkb | corvus: I responded to that comment. It doesn't play nice with filesystem log storage without a zuul build dashboard | 15:25 |
| *** zbr is now known as zbr|out | 15:25 | |
| *** jamesmcarthur has quit IRC | 15:25 | |
| clarkb | aiui we don't require the sql reporter yet so may have users that are using filesystem storage without the dashboard | 15:25 |
| *** mattw4 has quit IRC | 15:25 | |
| clarkb | (this is why I think it is object storage specific, though can be convinced that the dashboard is the future and as a result aim for that future rather than worrying too much about a corner case today) | 15:26 |
| corvus | clarkb: i take your response to be a rejection of making it the default, which is fine, i have no argument there. i'm merely saying that the *name* of the option shouldn't be "object storage". many folks are using a filesystem with the dashboard, and could utilize this. we shouldn't tell them not to by naming the option that way. | 15:27 |
| clarkb | I see | 15:27 |
| clarkb | (I think in my head I've got them conflated because I see this as only necessary for ceph, even swift doesn't need it ...) | 15:27 |
| clarkb | I'll make the update | 15:27 |
| *** hashar has joined #zuul | 15:29 | |
| *** jamesmcarthur has joined #zuul | 15:33 | |
| *** irclogbot_1 has joined #zuul | 15:38 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Add option for object store friendly log paths https://review.opendev.org/679145 | 15:38 |
| *** altlogbot_1 has joined #zuul | 15:40 | |
| corvus | clarkb: do you think the zuul_log_shard_build -> zuul_log_path_shard_build transition is necessary? can we just document that zuul_log_path_shard_build is relevant instead? | 15:44 |
| clarkb | corvus: This was something I ran into with testing (and I may just not understand how ansible vars work) but if I tried to pass zuul_log_path_object_storage through with a default it broke ansible. https://review.opendev.org/#/c/679145/5/roles/emit-job-header/tasks/main.yaml that broke | 15:46 |
| clarkb | corvus: maybe if I set the var on emit-job-header that gets automagically passed through to set-zuul-log-path-fact? | 15:46 |
| clarkb | (I may be thinking of this too much like function parameters and less like global variable?) | 15:46 |
| corvus | clarkb: i think the second thing is true | 15:47 |
| corvus | clarkb: so maybe that's worth a try | 15:47 |
| clarkb | k let me push up a chnge that only documents that var in emit-job-header | 15:47 |
| openstackgerrit | Merged zuul/zuul-jobs master: Document globally unique container name requirement https://review.opendev.org/679814 | 15:50 |
| *** dkehn has joined #zuul | 15:51 | |
| clarkb | (just sorting out cross reference links now) | 15:55 |
| clarkb | corvus: :rolevar:`set-zuul-log-fact-path.items.zuul_log_path_shard_build` fails with src/zuul/zuul-jobs/roles/emit-job-header/README.rst:19:Unknown interpreted text role "rolevar". | 15:58 |
| clarkb | I thought I did that correctly based on the zuul-sphinx docs. Any idea if I'm missing something? | 15:58 |
| clarkb | hrm maybe I nee to set the role:: value in the other README.rst? Or maybe we can't cross reference bteween Readmes? | 15:59 |
| corvus | clarkb: try dropping '.items.' ? | 16:00 |
| corvus | clarkb: (i'm guessing that based based on the generated anchor at: https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_45/679145/7/check/zuul-tox-docs/bbc3785/docs/log-roles.html#rolevar-set-zuul-log-path-fact.zuul_log_path_object_storage ) | 16:00 |
| corvus | xref between readmes should be fine | 16:00 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Add option for object store friendly log paths https://review.opendev.org/679145 | 16:18 |
| clarkb | I think Igot that figured out | 16:18 |
| corvus | \o/ | 16:19 |
| clarkb | testing should test if the var passthrough works as expected | 16:19 |
| *** zbr|out is now known as zbr | 16:19 | |
| *** noorul has joined #zuul | 16:32 | |
| noorul | hi | 16:32 |
| *** sshnaidm|ruck is now known as sshnaidm|bbl | 16:32 | |
| noorul | Anyone has a sample nginx config for hosting logs ? | 16:32 |
| noorul | Looking for way to uncompress gzipped files at server side | 16:33 |
| noorul | I wonder how new users of zuul solve this | 16:33 |
| noorul | ofosos: hi | 16:33 |
| corvus | clarkb, tobiash: do you know the status of the json appending log stuff? | 16:36 |
| tobiash | corvus: hasn't that landed already? | 16:36 |
| corvus | tobiash: great! did that have the change to correctly mark failed tasks? | 16:37 |
| noorul | corvus: I was going through http://lists.zuul-ci.org/pipermail/zuul-discuss/2019-August/000986.html | 16:37 |
| noorul | corvus: I am not able to see the console tab | 16:37 |
| tobiash | I think you're running it in production | 16:37 |
| tobiash | (like we do) | 16:37 |
| tobiash | corvus: I don't know recall about the other change | 16:38 |
| tobiash | But I think I remember landing such a change as well | 16:38 |
| clarkb | yes I restarted all the executors to pick up that change as well as the git repo updating fix | 16:38 |
| corvus | Set failed, unreachable, skipped statuses in json plugin | 16:38 |
| corvus | yep that merged too | 16:38 |
| corvus | i think there are 2 or 3 more changes related to log handling related to that: 1) update the js web ui console browser to recognize the new failed/skipped statuses; 2) fix and merge https://review.opendev.org/676723 3) update the js web ui console browser if needed by #2 (should not be necessary) | 16:40 |
| corvus | clarkb: once we complete #1, i think the console log browser may be sufficiently complete that opendev could disable ara in order to reduce the pressure on object stores | 16:41 |
| corvus | (nested aras would still do their thing of course) | 16:42 |
| clarkb | cool | 16:42 |
| clarkb | corvus: on the passing pars all the way through question I don't think that worked https://zuul.opendev.org/t/zuul/build/3c2575e646d74dba8d4abc6dd012f78c/log/job-output.txt#113 | 16:47 |
| clarkb | it used the default behavior there which failed the test | 16:47 |
| corvus | clarkb: amusingly, that illustrates the js console fix that we still need :) | 16:49 |
| corvus | https://zuul.opendev.org/t/zuul/build/3c2575e646d74dba8d4abc6dd012f78c/console#1/0/9/centos-7 | 16:49 |
| clarkb | corvus: have any other ideas on how we might avoid needing two different var names to pass that through? | 16:51 |
| corvus | clarkb: left comment on 679145 i think that may be the error | 16:51 |
| clarkb | aha thanks | 16:52 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Add option for object store friendly log paths https://review.opendev.org/679145 | 16:52 |
| *** jpena is now known as jpena|off | 16:53 | |
| -openstackstatus- NOTICE: Zuul job logs stored in OVH may fail. We have updated the base job to remove OVH from our storage location. If you have POST_FAILURES a recheck should fix them at this point. | 16:54 | |
| noorul | can anyone help me with nginx? | 16:54 |
| noorul | corvus: hi | 17:01 |
| corvus | noorul: i don't know anything about nginx. i wrote a bunch of documentation and examples for apache though. | 17:01 |
| noorul | corvus: ok | 17:02 |
| noorul | corvus: http://lists.zuul-ci.org/pipermail/zuul-discuss/2019-August/000986.html | 17:02 |
| noorul | corvus: I think I followed everything that is mentioned in that e-mail | 17:02 |
| noorul | corvus: But I still don't see the console tab | 17:02 |
| corvus | noorul: do you see the "results" header below the summary table? | 17:03 |
| noorul | I see results header | 17:04 |
| noorul | But nothing under that | 17:04 |
| noorul | https://pasteboard.co/IvSnUWD.png | 17:05 |
| *** electrofelix has quit IRC | 17:06 | |
| *** mattw4 has joined #zuul | 17:06 | |
| corvus | noorul: oh you have a logs tab, neat; what does that look like? | 17:06 |
| *** hashar has quit IRC | 17:07 | |
| noorul | https://pasteboard.co/IvSoGL1.png | 17:07 |
| corvus | noorul: both the results table and the console tab rely on "job-output.json" so you might want to take a look at that and see if the web server is returning it correctly (right content type, etc). try openinng it in your browser and see if it looks right. try opening up the developer tools in your browser and reloading the summary page and see if there's an error reported. | 17:09 |
| corvus | noorul: there could be a cors error, but the fact that your browser was able to load zuul-manifest.json suggests that's working. i'd expect it to work for job-output.json as well, but perhaps there's a subtle issue there. | 17:10 |
| noorul | corvus: It is not because it is stored as gzipped file | 17:11 |
| noorul | corvus: I have no clue on how to make nginx uncompress those files | 17:11 |
| noorul | corvus: Google is not helping me much | 17:11 |
| corvus | noorul: if it's gzipped, why does it appear as "job-output.json" instead of "job-output.json.gz" ? | 17:14 |
| noorul | http://10.29.12.161:8000/56/56/c0483f462c18c25cd4aff917f53f94e58fe1dd2e/check/testjob/a0744a8/job-output.json actually should return uncompressed fil | 17:16 |
| noorul | *file | 17:16 |
| noorul | But I get 404 | 17:16 |
| corvus | noorul: i can't visit a 10. url | 17:16 |
| corvus | that's an rfc1918 address | 17:16 |
| noorul | What I am trying to say is that, I haven't done anything special to change the link to job-output.json | 17:17 |
| corvus | noorul: i don't understand what is compressing your files then? | 17:17 |
| pabelanger | if using upload-logs role, that will compress | 17:18 |
| noorul | I am using that role | 17:18 |
| *** rfolco is now known as rfolco|bank | 17:18 | |
| pabelanger | which then needs more web server configuration, that's mostly why I created http://lists.zuul-ci.org/pipermail/zuul-discuss/2018-October/000565.html topic on ML before. | 17:19 |
| noorul | https://etherpad.openstack.org/p/rWL36RmF6W | 17:19 |
| noorul | line 170 | 17:19 |
| pabelanger | new users also needed to setup web config to via .gz files | 17:19 |
| corvus | oy | 17:19 |
| corvus | that's broken | 17:19 |
| corvus | someone please submit a patch to stop doing taht | 17:20 |
| pabelanger | but, I am not sure why job-output.json is uncompressed | 17:20 |
| corvus | pabelanger: because upload-logs runs after generate-zuul-manifest | 17:20 |
| pabelanger | Ah | 17:20 |
| corvus | so the manifest is wrong since it's compressed after its run | 17:20 |
| pabelanger | https://review.opendev.org/567696/ was my patch from a while back to allow it to be toggles in upload-logs | 17:21 |
| pabelanger | but maybe toggle isn't even needed now? | 17:21 |
| * noorul is lost | 17:21 | |
| noorul | In production I should be using compressed files right? | 17:22 |
| corvus | pabelanger: we need something -- either a toggle (regardless of the default) or to remove it altogether. clarkb was the only objection to that last time. let's see what he thinks. | 17:23 |
| corvus | but noorul is basically dead in the water until someone comes up with documentation for how to use the upload-logs role with nginx, or that change lands (or noorul decides to use apache instead) | 17:23 |
| clarkb | for opendev we get the compression out of our object stores so I don't have objections from the opendev side. I do think it is likely a worthwhile feature for people to opt into compression so a toggle is good but I can see how having it off by default would help people bootstrap | 17:24 |
| clarkb | corvus: https://review.opendev.org/#/c/679145/10 is failing tests now because it has a different base parent job than normal base and that still includes ovh in the swift target list | 17:25 |
| clarkb | any objection to me removing ovh globally so those jobs work too? | 17:25 |
| corvus | clarkb: no, sorry i missed that | 17:25 |
| corvus | clarkb, pabelanger: the really broken thing about this is that it compresses *after* the manifest generation, so the manifest is guaranteed to be wrong. that really rubs me the wrong way. | 17:26 |
| pabelanger | Yah, i didn't even realize that was an issue now too | 17:27 |
| corvus | the results table has a fallback where it will try ".json.gz" and ".gz", but the console tab does not | 17:27 |
| pabelanger | I think default uncompressed is a good new user experience, then if zuul operator wants compress, more configuration is needed | 17:27 |
| pabelanger | that was the original context of the ML post | 17:28 |
| pabelanger | which now seems to be what generate manifest role wants too | 17:28 |
| corvus | yeah, if compression is to happen, it should happen before generating the manifest | 17:28 |
| corvus | so that should really be another role | 17:29 |
| corvus | you would just lose a few more log entries in the output in that case | 17:29 |
| corvus | pabelanger: it sounds like clarkb is okay with the default change, so unless i'm misreading that, why don't you unwip your change, and send a 2 week announcement to zuul-announce? | 17:30 |
| pabelanger | wfm | 17:30 |
| corvus | noorul: do you have a way to make a local change to zuul-jobs? | 17:30 |
| noorul | corvus: which job? | 17:31 |
| corvus | noorul: the zuul-jobs repo | 17:31 |
| noorul | corvus: no, I am using opendev source | 17:31 |
| noorul | https://etherpad.openstack.org/p/rWL36RmF6W | 17:32 |
| noorul | line 11 | 17:32 |
| corvus | noorul: https://review.opendev.org/567696 will fix your situation, but we have to wait 2 weeks to merge it because we're changing the default behavior | 17:32 |
| corvus | noorul: if you can't wait that long, you could make a local fork of zuul-jobs and use that instead | 17:32 |
| noorul | corvus: I got it | 17:33 |
| noorul | corvus: But I need to still solve this for production | 17:33 |
| noorul | corvus: I don't think plain text will be ideal for production keeping size in mind | 17:33 |
| corvus | noorul: then you'll also want a role that runs before generate-zuul-manifest which compresses whatever you want before all this happens. i don't know if that role exists or if someone will need to write it. | 17:34 |
| corvus | noorul: but the sequence should be: (1) compress log files; (2) generate manifest; (3) upload log files | 17:35 |
| pabelanger | could upload-logs be updated to do generate manifest by default now? | 17:37 |
| pabelanger | then that be removed from post-run playbook | 17:37 |
| *** noorul has quit IRC | 17:40 | |
| Shrews | corvus: That gerrit email is exciting. Happy to lend a hand with whatever you need there when the time comes. | 17:41 |
| corvus | Shrews: \o/ writing up the subchecks design doc is next on my list -- i bet once that's done it'll be easier for you to pitch in | 17:42 |
| Shrews | I would also find it oddly satisfying if they used zuul to develop their gerrit, as we use their gerrit to develop our zuul | 17:43 |
| corvus | Shrews: omg that's going to make a great picture :) | 17:46 |
| Shrews | ikr? | 17:46 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Web improvement ideas https://review.opendev.org/680109 | 17:54 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Specify open source projects using Zuul https://review.opendev.org/680110 | 17:54 |
| clarkb | a couple ideas around maybe improving search engine juice for zuul-ci.org | 17:54 |
| clarkb | I also pushed https://review.opendev.org/#/c/679670/ recently to add an FAQ page which I think would be good pre ansiblefest | 17:56 |
| clarkb | in part because maybe we'll have concrete answers to some questions then :) | 17:56 |
| *** hashar has joined #zuul | 17:58 | |
| Shrews | clarkb: i think the faq is a good idea. Sort of feels hidden down in the Documentation menu. I wonder if it could/should be presented prominently on the home page as well? | 18:02 |
| Shrews | But at least having it somewhere feels like the rttd | 18:03 |
| clarkb | Shrews: ya Ithink we can add links to it whereever we like. Structurally it seemed like it belonged in the docs/ dir so that is how it ended up there in the current commit | 18:03 |
| Shrews | clarkb: yeah, i agree it belongs there with the other docs. but since it's usually the first thing I look for when researching some new whizbang, its nice to not have to hunt for it too much | 18:04 |
| Shrews | but yeah, can add links later | 18:05 |
| clarkb | Shrews: if you've got a suggestion for where a link would be appropriate (maybe in the footer too? though that isn't very prominent either) I'd be happy to try and put it in there | 18:12 |
| Shrews | clarkb: already left one | 18:12 |
| clarkb | great thanks | 18:12 |
| Shrews | it's pretty common, as far as i've seen, to have a link to the FAQ somewhere in the footer | 18:13 |
| Shrews | clarkb: also, left a comment on 680109 | 18:14 |
| clarkb | Shrews: responed. Did you see the tagline adjacent to the logo? | 18:16 |
| clarkb | (wondering if that was missed, could imply that isn't a prominent location as expected) | 18:17 |
| Shrews | clarkb: i did not | 18:18 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 18:18 |
| clarkb | Shrews: ^ faq footer link | 18:19 |
| Shrews | clarkb: the new phrase wraps for me on my laptop, so it doesn't look quite as good | 18:19 |
| clarkb | thats good feedback (doesn't wrap on my browser, but I'm new to this web stuff so don't yet know to test all the different sizes I guess) | 18:20 |
| clarkb | I'll likely defer to people that do webby things on this stuff. Was just fiddling with ideas on making text webcrawlers find more prominent | 18:21 |
| Shrews | been a while since i had any interest in web stuff, but i could have swore there was some section/magic incantation one could use have search words indexed, even if not visible... i might be misremembering | 18:21 |
| clarkb | I think titles are used for that (thought they aren't entirely invisible | 18:22 |
| Shrews | but, that was back in the days of <blink> | 18:22 |
| Shrews | :) | 18:22 |
| clarkb | the second change takes a different appraoch which is to add links (though I think it is more important to have links to you than out of you) as those should also tell indexers how connected we are? | 18:23 |
| clarkb | but as I said these were just ideas I was fiddling with and will defer to others who know more | 18:23 |
| clarkb | corvus: AJaeger https://review.opendev.org/#/c/679145/10 is ready for review now +1 from zuul | 18:27 |
| Shrews | clarkb: fwiw, i like the FAQ link in the footer. thx. i guess now we need some content :) | 18:29 |
| clarkb | ++ | 18:32 |
| clarkb | oh hrm I may need to edit the footer on all pages will do that later (doing house stuff now) | 18:32 |
| daniel2 | Something I brought up awhile ago. Using the docker nodepool builder image, qemu-img isnt there. Would I have to manually create a dockerfile to get that? | 18:38 |
| pabelanger | IMO, nodepool-builder should depend on diskimage-builder docker image, as qemu is a dependency of DIB | 18:39 |
| daniel2 | that actually sounds like a better option. | 18:39 |
| *** jamesmcarthur has quit IRC | 18:43 | |
| *** jamesmcarthur has joined #zuul | 18:44 | |
| daniel2 | pabelanger: Is there even a docker image for diskimage-builder? | 18:46 |
| pabelanger | I don't think so | 18:47 |
| pabelanger | in the case of docker quickstart, I don't think we support an option to build an image using nodepool-builder, we pull down another ubuntu image for it | 18:48 |
| pabelanger | possible tobiash has one? | 18:48 |
| AJaeger | corvus: happy now with https://review.opendev.org/679145 and https://review.opendev.org/679819 ? Those are clarkb's "swift" changes | 18:50 |
| *** irclogbot_1 has quit IRC | 18:54 | |
| daniel2 | pabelanger: Is it possible to run the builder outside of docker and have the launcher communicate with it? | 18:54 |
| pabelanger | daniel2: I belive so, data will be stored in zookeeper. As long as both have access to it, should be okay | 18:55 |
| *** irclogbot_1 has joined #zuul | 18:56 | |
| *** bhavikdbavishi has quit IRC | 19:00 | |
| *** bhavikdbavishi has joined #zuul | 19:02 | |
| *** irclogbot_1 has quit IRC | 19:02 | |
| Shrews | daniel2: launchers don't communicate with builders. launchers can *use* what the builders produce, but there is no communication otherwise | 19:02 |
| daniel2 | How would the launcher know what the builders produce then? | 19:03 |
| *** irclogbot_2 has joined #zuul | 19:04 | |
| Shrews | daniel2: by image name in the config files | 19:05 |
| daniel2 | ah okay | 19:07 |
| Shrews | launchers are told to look for an image named XYZ in the provider, but don't care how image XYZ is put there (example: https://opendev.org/openstack/project-config/src/branch/master/nodepool/nl01.openstack.org.yaml#L172) | 19:08 |
| Shrews | daniel2: ^^ | 19:08 |
| Shrews | i guess technically that section means it comes from the builder, but they don't communicate anything about it :/ | 19:14 |
| *** bhavikdbavishi has quit IRC | 19:16 | |
| *** jamesmcarthur has quit IRC | 19:23 | |
| daniel2 | Curious, could nodepool build on top of an openstack image? Basically boot the openstack image, do stuff on it, and snapshot it or something? | 19:25 |
| pabelanger | it used to work that way | 19:25 |
| pabelanger | but we removed it for nodepool refactor | 19:25 |
| pabelanger | I think the plan was open to add it back, some somebody needed do that | 19:25 |
| daniel2 | How does it work with ansible now though? | 19:26 |
| daniel2 | I heard that replaced the original method | 19:26 |
| pabelanger | we'll, we've also talked about an ansible based build too, I'd be interested in that, but haven't had time to dig more into it | 19:26 |
| corvus | AJaeger, clarkb: swift changes lgtm | 19:27 |
| pabelanger | our use case are network vendor images, like cisco ios or juniper junos | 19:27 |
| clarkb | pabelanger: daniel2 note that disk image builder's ubuntu element supports modifying the default upstream cloud image by default | 19:27 |
| pabelanger | we use a custom ansible playbook to bootstrap them, and update into clouds | 19:27 |
| clarkb | it is only the ubuntu-minimal element that does debootstrap from scratch | 19:27 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 19:29 |
| clarkb | Shrews: ^ that fixes the faq links on the other pages | 19:29 |
| daniel2 | i feel like building the images for us is a little much as we dont need anything different from a base ubuntu image. | 19:31 |
| daniel2 | We just need images we can install devstack on and run tests. | 19:31 |
| clarkb | daniel2: in that case you can tell nodepool to use images that already exist in a cloud | 19:31 |
| Shrews | daniel2: cloud-images config option is probably what you want then | 19:31 |
| daniel2 | We have our own openstack cluster in house, that is what I was curious about earlier. | 19:31 |
| daniel2 | Can you point me to the documentation for that? | 19:32 |
| Shrews | daniel2: https://zuul-ci.org/docs/nodepool/configuration.html#attr-providers.[openstack].cloud-images | 19:33 |
| daniel2 | thank you | 19:33 |
| *** jamesmcarthur has joined #zuul | 19:33 | |
| Shrews | daniel2: if you need an example, this is a config used in our nodepool test suite: https://opendev.org/zuul/nodepool/src/branch/master/nodepool/tests/fixtures/node_unmanaged_image.yaml | 19:39 |
| AJaeger | pabelanger, mnaser, want to review clarkb's "swift" changes for logs https://review.opendev.org/#/c/679145/ and https://review.opendev.org/#/c/679819/ ? Otherwise I'll approve at the hour... | 19:43 |
| clarkb | AJaeger: thanks | 19:43 |
| *** bjackman_ has quit IRC | 19:44 | |
| *** bjackman_ has joined #zuul | 19:47 | |
| pabelanger | +2 | 19:47 |
| pabelanger | looks backwards compat | 19:47 |
| clarkb | pabelanger: the test should confirm that (via the default run of the role, you can also confirm it via the output of that play) | 19:48 |
| clarkb | (I checked it myself and ya seems to be correct | 19:48 |
| pabelanger | yah, looks there too! Thanks for adding them | 19:48 |
| clarkb | in fact when I got the var name wrong it did the default twice and failed the first check so that seems to be working happily :) | 19:49 |
| *** jamesmcarthur has quit IRC | 19:54 | |
| *** rfolco|bank is now known as rfolco | 19:55 | |
| * AJaeger approved the zuul-jobs change, will +A opendev/base-jobs once the first is merged... | 20:00 | |
| *** jamesmcarthur has joined #zuul | 20:08 | |
| *** hashar has quit IRC | 20:09 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Fix "Set failed, unreachable, skipped statuses in json plugin" https://review.opendev.org/680159 | 20:14 |
| corvus | clarkb: ^ this is what we get for merging changes without tests :( | 20:15 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add option for object store friendly log paths https://review.opendev.org/679145 | 20:15 |
| corvus | unfortunately, we're going to need to wait for that to land, and then deploy it before i can make progress on the javascript to actually use it :( | 20:15 |
| clarkb | k | 20:16 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Fix "Set failed, unreachable, skipped statuses in json plugin" https://review.opendev.org/680159 | 20:17 |
| AJaeger | pabelanger: could you review this RHEL8 related change, please? https://review.opendev.org/679126 | 20:18 |
| AJaeger | pabelanger: and this one as well: https://review.opendev.org/674092 | 20:18 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 20:20 |
| clarkb | Shrews: ^ I added content for things I've been getting asked foundation side for ansiblefest prep | 20:21 |
| pabelanger | AJaeger: I'm not the best to review there, don't have access to RHEL8. maybe tristanC can helpout | 20:21 |
| pabelanger | or dmsimard | 20:21 |
| Shrews | clarkb: nice. lgtm, except maybe we should mention AWS nodepool support too | 20:25 |
| clarkb | pabelanger: AJaeger maybe we can get software factory to report ci results on that? | 20:25 |
| clarkb | Shrews: ++ also is there gce work in progress? | 20:25 |
| pabelanger | AJaeger: did leave comment on 674092 | 20:25 |
| pabelanger | we (zuul.a.c) want to run rhel8, just haven't set it up yet | 20:25 |
| Shrews | clarkb: not that i'm aware of. there is an azure driver up for review | 20:25 |
| pabelanger | but SF.io does sound good too | 20:26 |
| Shrews | oh, you mentioned azure | 20:26 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 20:27 |
| clarkb | Shrews: ^ I split the paragraphs and alpha sorted them too | 20:27 |
| Shrews | clarkb: lgtm. should the FAQ link in the footer also be present on the FAQ page itself? it's redundant to do so, but consistent | 20:35 |
| Shrews | maybe the faq doesn't need the footer | 20:36 |
| clarkb | ya I'll add it to be consistent | 20:36 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 20:37 |
| Shrews | +2 | 20:37 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: DO NOT MERGE test log path changes https://review.opendev.org/680178 | 20:46 |
| clarkb | jamesmcarthur: re the css question I don't know. | 20:49 |
| clarkb | the css seems to set a list-style to disc on ul items | 20:50 |
| clarkb | I don't know why we don't get those | 20:50 |
| clarkb | maybe because it is in a p? | 20:51 |
| clarkb | no moving that around didn't change anything. Fixing that might require someone with more css knowledge than me | 20:52 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 20:53 |
| clarkb | jamesmcarthur: ^ fixed the missing "be" but not the css | 20:53 |
| jamesmcarthur | clarkb: got it! | 20:54 |
| clarkb | will a web browser debugger identify what css directives apply to an element? | 20:55 |
| jamesmcarthur | the issue is b/c it's included in line 18 on main.css | 20:55 |
| jamesmcarthur | ul and li are both set to padding: 0 and margin: 0 | 20:56 |
| jamesmcarthur | I can have a better look at it when I work on the meta tags | 20:56 |
| clarkb | ha testing the log path changes we don't actually get to see if those took affect if we upload logs to rax because forthe public access its all HMAC'd cdn there | 20:57 |
| jamesmcarthur | and then on line 32, the list-style is overridden to set it to none | 20:57 |
| jamesmcarthur | ol, ul { | 20:57 |
| jamesmcarthur | list-style: none; } | 20:57 |
| clarkb | does line 1641 not override that? | 20:58 |
| jamesmcarthur | no, b/c that applies to a specific ul with the class name "default" - ul.default | 20:59 |
| jamesmcarthur | If you remove ol/ul from line 16 and then delete line 32 it resolves the issue, at least on the FAQ | 20:59 |
| jamesmcarthur | but it may have repercussions elsewhere | 21:00 |
| jamesmcarthur | i need to download it and review on my local machine | 21:00 |
| jamesmcarthur | I'm happy to put a patch through | 21:00 |
| *** sshnaidm|bbl is now known as sshnaidm|afk | 21:01 | |
| clarkb | feel free to push an update to my change | 21:01 |
| clarkb | or a follow on change | 21:01 |
| jamesmcarthur | roger that | 21:01 |
| Shrews | zuul-maint: FYI, there exists a not-zero chance of me losing power tomorrow due to Dorian. Not expecting high sustained winds here, but certainly high gusts which is usually enough to affect our crappy power structure in my neighborhood. | 21:03 |
| corvus | Shrews: stay safe -- and if you do lose power, maybe we should all take the day off in solidarity | 21:05 |
| Shrews | corvus: i approve of this unifying stance | 21:05 |
| clarkb | corvus: I like that idea :) | 21:07 |
| corvus | zuul-maint: if someone else could +3 https://review.opendev.org/680159 that would be great -- then we can restart the opendev executors and i can finish up development of the console browser | 21:08 |
| Shrews | looking | 21:09 |
| Shrews | corvus: +3 | 21:12 |
| *** jamesmcarthur has quit IRC | 21:25 | |
| *** jamesmcarthur has joined #zuul | 21:27 | |
| *** jamesmcarthur has quit IRC | 21:30 | |
| *** jamesmcarthur has joined #zuul | 21:30 | |
| *** bjackman_ has quit IRC | 21:43 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Only use RHEL8 deps repo on Red Hat systems newer than 7 https://review.opendev.org/679126 | 21:57 |
| openstackgerrit | Merged zuul/zuul master: Fix "Set failed, unreachable, skipped statuses in json plugin" https://review.opendev.org/680159 | 22:02 |
| corvus | https://gerrit-review.googlesource.com/q/topic:%2522subchecks%2522 | 22:03 |
| corvus | there's the draft design doc for gerrit sub-checks | 22:03 |
| *** jamesmcarthur has quit IRC | 22:07 | |
| *** jamesmcarthur has joined #zuul | 22:10 | |
| *** jhesketh has quit IRC | 22:27 | |
| *** jhesketh has joined #zuul | 22:33 | |
| *** dmsimard1 has joined #zuul | 22:47 | |
| *** dmsimard has quit IRC | 22:47 | |
| *** dmsimard1 is now known as dmsimard | 22:50 | |
| *** tosky has quit IRC | 23:02 | |
| *** threestrands has joined #zuul | 23:17 | |
| *** jamesmcarthur has joined #zuul | 23:17 | |
| *** jamesmcarthur has quit IRC | 23:18 | |
| *** armstrongs has joined #zuul | 23:34 | |
| *** jamesmcarthur has joined #zuul | 23:40 | |
| *** armstrongs has quit IRC | 23:44 | |
| *** nborg1 has quit IRC | 23:48 | |
| *** mattw4 has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
