openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 00:41 |
---|---|---|
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 00:48 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 00:52 |
mnaser | avass: woops i didnt see that because of the typo in my name :p | 00:54 |
mnaser | yeah, i mean i'm open to whatever, but it seems to fit the theme of these days :) | 00:54 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 01:01 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 01:06 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 01:14 |
*** rfolco has joined #zuul | 01:24 | |
*** swest has quit IRC | 01:28 | |
*** ysandeep|away is now known as ysandeep|rover | 01:30 | |
*** ysandeep|rover is now known as ysandeep|away | 01:31 | |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 01:33 |
*** swest has joined #zuul | 01:44 | |
*** rfolco has quit IRC | 01:45 | |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 01:57 |
*** Goneri has quit IRC | 02:15 | |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 02:44 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 02:52 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 03:03 |
*** ysandeep|away is now known as ysandeep|rover | 03:33 | |
*** bhavikdbavishi has joined #zuul | 03:57 | |
*** bhavikdbavishi1 has joined #zuul | 04:02 | |
*** bhavikdbavishi has quit IRC | 04:04 | |
*** bhavikdbavishi1 is now known as bhavikdbavishi | 04:04 | |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 04:27 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Add bionic-plain testing https://review.opendev.org/717655 | 04:27 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: test jobs: fixup autogeneration header https://review.opendev.org/717656 | 04:27 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 04:29 |
*** evrardjp has quit IRC | 04:36 | |
*** evrardjp has joined #zuul | 04:37 | |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Add bionic-plain non-voting testing https://review.opendev.org/717655 | 05:44 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: test jobs: fixup autogeneration header https://review.opendev.org/717656 | 05:44 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 05:44 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 05:44 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-tox: preinstall pip https://review.opendev.org/717663 | 05:44 |
*** dpawlik has joined #zuul | 06:28 | |
*** rpittau|afk is now known as rpittau | 06:35 | |
*** bhavikdbavishi has quit IRC | 06:37 | |
*** bhavikdbavishi has joined #zuul | 06:37 | |
tobiash | clarkb: re leaked dib processes: I've found one as well. What's weird about that is that it has been dropped from nodepool in the process hierarchy. What I also found is that this image faced a build timeout in nodepool. So maybe canceling timeouted builds in nodepool has an issue. | 06:41 |
*** avass is now known as Guest62808 | 06:44 | |
*** avass has joined #zuul | 06:44 | |
*** bhavikdbavishi has quit IRC | 06:45 | |
*** saneax has joined #zuul | 06:53 | |
*** yolanda has joined #zuul | 06:56 | |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: test jobs: fixup autogeneration header https://review.opendev.org/717656 | 06:57 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 06:57 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-tox: preinstall pip https://review.opendev.org/717663 | 06:57 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 06:58 |
openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: update Python version documentation https://review.opendev.org/717669 | 07:05 |
*** jcapitao has joined #zuul | 07:15 | |
openstackgerrit | Merged zuul/zuul-jobs master: ensure-tox: update Python version documentation https://review.opendev.org/717669 | 07:24 |
*** gtema has joined #zuul | 07:49 | |
*** bhavikdbavishi has joined #zuul | 07:51 | |
*** bhavikdbavishi1 has joined #zuul | 07:54 | |
*** bhavikdbavishi has quit IRC | 07:55 | |
*** bhavikdbavishi1 is now known as bhavikdbavishi | 07:55 | |
*** tosky has joined #zuul | 07:58 | |
*** spsurya_ has joined #zuul | 07:59 | |
frickler | so what's with these "Comments left for invalid file" in https://review.opendev.org/717533 ? | 08:04 |
frickler | would be great if one could at least see from which job they come instead of having to check them all | 08:04 |
*** gtema has quit IRC | 08:05 | |
*** gtema has joined #zuul | 08:06 | |
*** gtema has quit IRC | 08:06 | |
*** gtema has joined #zuul | 08:07 | |
*** sshnaidm|afk is now known as sshnaidm | 08:07 | |
AJaeger | frickler: they come from https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/library/tox_parse_output.py - the work that tobiash, mnaser, mordred did. | 08:14 |
AJaeger | I think the invalid file is a reaction of gerrit to notify about a file that is not in the commit - but I'm not completly sure about that. | 08:16 |
frickler | AJaeger: that's what I'm assuming, yes, therefore I'm asking whether we can make those comments more easily consumable, otherwise zuul should stop submitting them IMO | 08:17 |
AJaeger | frickler: I proposed in https://review.opendev.org/#/c/717486/ to check which files are part of the change and only report on those (iterate over list of files in git change) - 717486 was a quick fix | 08:19 |
frickler | AJaeger: that was twice the same patch for ignoring .tox, I cannot find the other one | 08:27 |
AJaeger | frickler: it was meant to be the same | 08:27 |
AJaeger | The commit message explained what was the better solution but I did the quick fix only | 08:28 |
*** dpawlik has quit IRC | 08:29 | |
*** ysandeep|rover is now known as ysandeep|lunch | 08:36 | |
*** dpawlik has joined #zuul | 08:38 | |
*** ysandeep|lunch is now known as ysandeep | 09:18 | |
*** ysandeep is now known as ysandeep|rover | 09:19 | |
openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Add support for ansible callbacks https://review.opendev.org/717260 | 09:31 |
openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 09:36 |
openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 09:37 |
*** gtema has quit IRC | 10:32 | |
*** bhavikdbavishi has quit IRC | 10:54 | |
*** bhavikdbavishi has joined #zuul | 11:00 | |
*** gtema has joined #zuul | 11:03 | |
*** jcapitao is now known as jcapitao_lunch | 11:05 | |
openstackgerrit | Jan Kundrát proposed zuul/zuul-jobs master: docs: fix a typo in `run-test-command` https://review.opendev.org/717713 | 11:18 |
*** bhavikdbavishi has quit IRC | 12:11 | |
*** rlandy has joined #zuul | 12:12 | |
*** jcapitao_lunch is now known as jcapitao | 12:25 | |
openstackgerrit | Fabien Boucher proposed zuul/zuul master: pagure: Make use of the new project webhook/token endpoint https://review.opendev.org/717732 | 12:30 |
*** Goneri has joined #zuul | 12:31 | |
*** rfolco has joined #zuul | 12:35 | |
mnaser | AJaeger, frickler: we're working on cleaning all that up.. | 12:44 |
AJaeger | mnaser: thanks! who's we? | 12:45 |
mnaser | AJaeger: i think the collective we of zuul-ers as we discover more "invalid file" comments :p | 12:45 |
AJaeger | mnaser: Ah ;) | 12:46 |
openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 13:03 |
*** hashar has joined #zuul | 13:06 | |
*** Goneri has quit IRC | 13:08 | |
zbr | mnaser: clarkb: i suppose today we merge the s/install/ensure/ stuff, right? | 13:13 |
zbr | asking because i want to see them in as i have several other patches that would need to be rebased after that. | 13:14 |
fungi | today's when we announced we would given no objections, if i'm remembering accurately | 13:14 |
*** Goneri has joined #zuul | 13:15 | |
*** avass is now known as Guest40054 | 13:24 | |
*** Guest62808 is now known as avass | 13:24 | |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: config: add environment variable substitution https://review.opendev.org/710644 | 13:25 |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add zuul-registry deployment https://review.opendev.org/710650 | 13:30 |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Update dhall-kubernetes version to v4.0.0 https://review.opendev.org/717342 | 13:42 |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add zuul-registry deployment https://review.opendev.org/710650 | 13:42 |
AJaeger | zbr: yes, I'll slowly start merging that one now | 13:52 |
AJaeger | config-core, infra-root, I slowly start merging the stack starting at https://review.opendev.org/#/c/716663 for renaming from roles from install to ensure. This should not cause problems - but let's play it safe... | 13:53 |
tristanC | mordred: heh, so opendev is going to deploy zuul using docker-compose ? ( i just saw https://review.opendev.org/717620 ) | 13:53 |
mordred | tristanC: it's how we're doing other container things atm - so I worked it up keeping the same pattern | 13:56 |
mordred | we haven't actually had an opendev-level conversation about updating the zuul deployment yet- I was just a bit bored this weekend | 13:57 |
*** zxiiro has joined #zuul | 13:57 | |
tristanC | mordred: oh ok, then i guess you gave up on using k8s? | 13:58 |
avass | AJaeger: oh, nice | 13:59 |
*** cdearborn has joined #zuul | 14:00 | |
mordred | tristanC: I don't know that we gave up on k8s as much as we haven't reached a point where we've decided to try using k8s in the first place ;) | 14:02 |
mordred | tristanC: so far it hasn't been a road we've been ready to go down because it involves adding "running a k8s" to the list of things we need to do. at least so far, nothing we've been running in containers would be made so much easier by k8s that the cost of running k8s would be worth it. | 14:04 |
*** lennyb has quit IRC | 14:04 | |
*** gtema has quit IRC | 14:10 | |
*** gtema has joined #zuul | 14:11 | |
tristanC | mordred: ok fair enough. though it is surprising to see yet another zuul deployment recipe | 14:11 |
tristanC | mordred: if you are bored, those zuul-operator changes are waiting for review https://review.opendev.org/707193 and https://review.opendev.org/710650 | 14:12 |
mordred | tristanC: fair - I don't think we're aiming to make this one re-usable by others as much as "stop using puppet" :) | 14:13 |
*** sshnaidm is now known as sshnaidm|afk | 14:24 | |
mordred | tristanC: in other news though: https://review.opendev.org/#/c/717057/ - we're almost done getting rid of our cron job! | 14:25 |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: config: add environment variable substitution https://review.opendev.org/710644 | 14:26 |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Use explicit provides/requires for container jobs https://review.opendev.org/717767 | 14:26 |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add zuul-registry deployment https://review.opendev.org/710650 | 14:27 |
tristanC | mordred: iiuc the cron job is what applies puppet without being driven by ansible? | 14:27 |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-docker to ensure-docker for consistency https://review.opendev.org/716663 | 14:30 |
*** ysandeep|rover is now known as ysandeep|afk | 14:31 | |
*** bhavikdbavishi has joined #zuul | 14:33 | |
shanemcd | Hi everyone. I recently started seeing "Something went wrong." when trying to look at live logs in Safari. | 14:34 |
shanemcd | Not sure if this is helpful: https://imgur.com/a/YT99B5O | 14:34 |
shanemcd | Seems ok in other browsers I've tried | 14:34 |
*** bhavikdbavishi1 has joined #zuul | 14:35 | |
tristanC | shanemcd: it's probably related to the Content-Security-Policy setup by software-factory. for somereason, safari is more picky about it. i think we can fix that server side | 14:35 |
AJaeger | avass: will you update other repos as well? For example, check http://codesearch.openstack.org/?q=name%3A%20install-docker&i=nope&files=&repos= | 14:36 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: docker-install: workaround for centos-8 conflicts https://review.opendev.org/703053 | 14:36 |
avass | AJaeger: sure I'll take a look at it later | 14:37 |
*** bhavikdbavishi has quit IRC | 14:37 | |
*** bhavikdbavishi1 is now known as bhavikdbavishi | 14:37 | |
AJaeger | thanks, avass ! | 14:37 |
mordred | tristanC: the cron job is what currently runs ansible. ansible drives everything for us - including both native ansible things as well as puppet | 14:38 |
mordred | tristanC: those updates shift us to having ansible driven by zuul \o/ | 14:38 |
tristanC | mordred: oh i see, thus you are replacing cron with zuul? | 14:38 |
mordred | yup | 14:39 |
fungi | mordred: we *did* try deploying gitea in kubernetes, right? | 14:45 |
fungi | so not entirely fair to say we haven't tried yet | 14:45 |
tristanC | fungi: i do remember discussion about using ceph on k8s too | 14:45 |
mordred | yeah - but that's also a bit of a special case | 14:45 |
mordred | the gitea needs for that k8s deployment were quite intensive | 14:45 |
mordred | I'm not sure that's indicative of what a more "normal" use might look like | 14:46 |
mordred | fungi: but that it definitely true | 14:46 |
*** bhavikdbavishi has quit IRC | 14:46 | |
fungi | except the hope was that kubernetes would make it easier to deploy and manage a complex service like gitea. for "normal" (simple) services it seems like swatting a fly with a buick | 14:47 |
*** sshnaidm|afk is now known as sshnaidm | 14:51 | |
*** sgw has quit IRC | 14:53 | |
*** bhavikdbavishi has joined #zuul | 14:55 | |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-kubernetes to ensure-kubernetes for consistency https://review.opendev.org/716667 | 14:56 |
mordred | fungi: well - that wasn't the 'extra complex' part I mean. the running ceph and hooking it in as a k8s storage provider was the bit that made things a bit more heavyweight ... and even then, honestly, it didn't do a bad job, it was just figuring out how to get rook working for us took a while :) | 14:57 |
fungi | got it | 14:57 |
openstackgerrit | Monty Taylor proposed zuul/zuul master: Install skopeo in executors https://review.opendev.org/717613 | 15:04 |
openstackgerrit | Monty Taylor proposed zuul/zuul master: Add gear to the ansible envs https://review.opendev.org/717616 | 15:04 |
*** jcapitao is now known as jcapitao_afk | 15:05 | |
*** sgw has joined #zuul | 15:12 | |
*** ysandeep|afk is now known as ysandeep | 15:13 | |
clarkb | for us I think the biggest unsolved problem is simply operating a k8s | 15:16 |
clarkb | turns out the hosted options aren't great, and then we've tried a couple versions of the self managed k8s | 15:17 |
clarkb | and docker compose has been a really good transition point because the underlying structure doesn't change, just the "packaging" really | 15:18 |
clarkb | and from there you can take the same images then change the structure | 15:18 |
tristanC | clarkb: that makes sense. thanks for the details. | 15:20 |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-openshift to ensure-openshift for consistency https://review.opendev.org/716675 | 15:20 |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-podman to ensure-podman for consistency https://review.opendev.org/716682 | 15:20 |
tristanC | makes me wonder if such docker-compose based scenario could be implemented in a zuul project so that it can be re-used by other admin facing the same issue? | 15:21 |
clarkb | tobiash: interesting, ours don't ever seem to hit the build timeout according to logs | 15:21 |
clarkb | tobiash: my theory is/was that we are hitting a subprocess wait() call deadlock as warned against in the subprocess module docs | 15:21 |
*** gtema has quit IRC | 15:22 | |
mordred | tristanC: I think it would not be hard to extract what I've started on there once we're happy with it | 15:29 |
mordred | tristanC: I kind of purposefully didn't parameterize a few things just for expediency | 15:29 |
mordred | but - as a followup if we decided it was valuable, I don't think it would be hard to accomplish | 15:29 |
mordred | tristanC: could also see about adding it as an option to windmill ... since what we've got there is really just using compose as a replacement for pip/apt/rpm and systemd - the rest if all ansible | 15:30 |
mordred | (again, I didn't want to overcomplicate things this time) | 15:33 |
corvus | mordred, tristanC, mnaser: i left a note on https://review.opendev.org/717616 -- i just want to be clear that i don't think "opendev uses this" is necessarily a sufficient reason to add something to the zuul images. but if we still want to add it, i'm ok. | 15:34 |
tristanC | mordred: that sounds like a good plan. Shouldn't windmill be moved to the zuul/ namespace then? | 15:36 |
*** jcapitao_afk is now known as jcapitao | 15:37 | |
*** ysandeep is now known as ysandeep|away | 15:39 | |
mordred | tristanC: well - so far it's been paul - if we get to a point where we're looking at doing that and supporting it more directly we could certainly talk about it - but I think one thing at a time ;) | 15:42 |
mordred | corvus: I added "remove zuul-migrate" to https://etherpad.openstack.org/p/zuulv4 - seems like maybe a good time to cut that loose? | 15:43 |
corvus | mordred: heh, yeah :) | 15:44 |
tristanC | mordred: opendev system-config not leveraging/contributing to windmill doesn't help either... i kind of hoped puppet-zuul would have been replaced by one of the existing zuul deployment project. | 15:47 |
*** bhavikdbavishi has quit IRC | 15:52 | |
tobiash | tristanC, corvus: there is already a way to install more things into the ansible venvs (ANSIBLE_EXTRA_PACKAGES env var when running zuul-manage-ansible) | 16:00 |
mordred | tobiash: yeah - but that's run at container build time | 16:00 |
tobiash | a zuul.conf entry might confuse especially users who use the official docker images as the ansible vens are created during build time, not runtime | 16:00 |
tobiash | mordred: configuring via zuul.conf would be the same | 16:01 |
mordred | tobiash: oh - I might be responding to a different thing ... what's the context of your thing? | 16:01 |
tobiash | https://review.opendev.org/717616 | 16:01 |
mordred | tobiash: ah! I hadn't seen the comments there | 16:03 |
mordred | yeah - I think this is a build-time vs runtime thing - and a zuul.conf entry wouldn't help so much | 16:03 |
tobiash | so actually I see two ways to put extra stuff there for container users, add into the official image or create a new image based from the official one and re-run zuul-manage-ansible there | 16:03 |
mordred | yah | 16:03 |
tobiash | I guess the question is which to choose for gear | 16:04 |
tristanC | corvus: this wasn't part of the operator spec, but https://review.opendev.org/#/c/710650/ adds a registry component to deploy the zuul-registry. and it seems to work, integration tests shows `Login Succeeded!` | 16:04 |
tobiash | but at least there seem two users needing this | 16:04 |
mordred | yeah - I mean - it's a small depend, and it's already installed by zuul components anyway, so it's not a _big_ burden. otoh - process-wise it's maybe a little odd if we don't have the consuming code for it in zuul-jobs | 16:05 |
tristanC | mordred: gear almost made it into zuul-jobs with https://review.opendev.org/#/c/537847/ | 16:06 |
*** rpittau is now known as rpittau|afk | 16:08 | |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-devstack to ensure-devstack for consistency https://review.opendev.org/716685 | 16:08 |
mordred | tristanC: maybe we should revive that? clarkb ^^ (we have more of a pattern now for putting in some roles that require more infrastructure around them along with some docs on how to set that infrastructure up - like zuul-registry) | 16:10 |
clarkb | mordred: tristanC I never even knew that was a thing. I think its a fine idea. I also really don't see the tooling openstack uses as openstack specific | 16:11 |
clarkb | yes it was made to meet openstack's demands but nothing about the indexing pipeline is openstack specific | 16:11 |
clarkb | On the visualization side there are openstack things that need to be decoupled, but the indexing backend is literally "take logs and index tehm" | 16:12 |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-javascript-packages to ensure-javascript-packages for consistency https://review.opendev.org/716687 | 16:17 |
mnaser | corvus: yes, i agree with you, it's a +1 from me in the perspective of useful for zuul deployers and not opendev | 16:26 |
fungi | well, useful for opendev too | 16:32 |
fungi | we could potentially align with it at some point | 16:32 |
corvus | mnaser, mordred, tobiash, tristanC, clarkb, fungi: okay, i just didn't want to establish an "opendev needs it so we'll add it to the image" precedent. but it seems like we feel it's generally useful and low-cost enough to add it. i'm happy with the outcome. | 16:35 |
tobiash | ++ | 16:35 |
*** bhavikdbavishi has joined #zuul | 16:36 | |
corvus | mordred: though, apparently zuul has other thoughts :) https://zuul.opendev.org/t/zuul/build/d176984ce7814f91a319938601856a95 | 16:36 |
*** evrardjp has quit IRC | 16:36 | |
corvus | mordred: ah, it's the skopeo change that's failing | 16:36 |
*** evrardjp has joined #zuul | 16:37 | |
*** bhavikdbavishi1 has joined #zuul | 16:41 | |
*** bhavikdbavishi has quit IRC | 16:42 | |
*** bhavikdbavishi1 is now known as bhavikdbavishi | 16:42 | |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-nodejs to ensure-nodejs for consistency https://review.opendev.org/716692 | 16:50 |
clarkb | avass: tristanC https://review.opendev.org/#/c/716298/2 and https://review.opendev.org/#/c/708871/ conflcit with each other. Is there a prefered merge order? | 16:53 |
clarkb | tristanC: I left a question/note on https://review.opendev.org/#/c/716298/2 | 16:54 |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-yarn to ensure-yarn for consistency https://review.opendev.org/716693 | 16:56 |
clarkb | AJaeger: ^ fyi since you asked about reviews on those | 16:56 |
openstackgerrit | Tobias Henkel proposed zuul/zuul master: Defer setting build pause to event queue https://review.opendev.org/712939 | 16:57 |
mordred | corvus: oh. blerg. gpg isn't installed | 16:58 |
*** jcapitao has quit IRC | 16:58 | |
mordred | corvus: I feel like we fixed this somewhere recently | 16:58 |
mordred | but it certainly seems like the sort of thing that shoudl be installed in the excutor :) | 16:58 |
corvus | mordred: we fixed it in virtual machine images | 16:58 |
mordred | ah | 16:58 |
*** hashar is now known as hasharAway | 16:58 | |
clarkb | mordred: gnupg2 or gnupg are the packages on debuntu. The first gets you the gpg2 aliases and is transitional | 16:59 |
AJaeger | thanks, clarkb | 17:01 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: docker-install: workaround for centos-8 conflicts https://review.opendev.org/703053 | 17:03 |
openstackgerrit | Monty Taylor proposed zuul/zuul master: Add gear to the ansible envs https://review.opendev.org/717616 | 17:03 |
tobiash | mordred: commented ^ | 17:05 |
tristanC | clarkb: thanks, i replied. no prefered merge order. I guess adding kubectl support to zuul-base-jobs can wait | 17:12 |
AJaeger | zuul-jobs experts, there's a larger refactor of our jobs on how we handle repositories by mnaser, please review the stack starting at https://review.opendev.org/#/c/717507/2 | 17:13 |
clarkb | tristanC: avass ok both chagnes lgtm but I didn't approve in case there was a strong opinion on ordering | 17:14 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on ensure-podman https://review.opendev.org/716578 | 17:14 |
*** rlandy is now known as rlandy|lunch | 17:14 | |
mordred | tobiash: ++ | 17:15 |
mnaser | AJaeger: thanks for that heads up for others :) | 17:15 |
mnaser | i think that'll be an excellent cleanup/improvement | 17:15 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 17:18 |
openstackgerrit | Monty Taylor proposed zuul/zuul master: Add gear to the ansible envs https://review.opendev.org/717616 | 17:18 |
tobiash | mordred, corvus: do we want that different refresh button in https://review.opendev.org/716305 ? | 17:19 |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-pdk-dependencies to ensure-pdk-dependencies for consistency https://review.opendev.org/716695 | 17:20 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on ensure-podman https://review.opendev.org/716578 | 17:20 |
mordred | tobiash: I think so far folks have been ok with the change - but I think I'd be fine if someone knew how to make it look the other way (I don't) | 17:20 |
AJaeger | mnaser: did you test that with some depends-on? | 17:21 |
tobiash | I don't either ;) | 17:21 |
mordred | tobiash: :) | 17:21 |
*** gtema has joined #zuul | 17:23 | |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on ensure-podman https://review.opendev.org/716578 | 17:24 |
corvus | tobiash: yeah, i can live with it for now :) | 17:25 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on ensure-podman https://review.opendev.org/716578 | 17:27 |
*** gtema has quit IRC | 17:27 | |
mnaser | AJaeger: i think a lot of those are self testing inside zuul-jobs | 17:32 |
tobiash | mordred: is there some issue with an opendev mirror? I've seen this failure more than once today: https://fceab12b277662d76f26-6eadae60e9a1b03c6789f42d51c17d3a.ssl.cf1.rackcdn.com/717602/1/gate/zuul-tox-remote/1fbac99/job-output.txt | 17:35 |
clarkb | tobiash: thats downloading from node's source repo | 17:37 |
clarkb | tobiash: not our mirror | 17:37 |
clarkb | basically the package is being downloaded from the mirror successfully then we install the package and that triggers a download of node itself. I've looked into making that more reliable but they hard code the source location in the package | 17:37 |
clarkb | so there isn't really anything we can do unless we use a transparent proxy (which we don't due to potential for abuse) | 17:38 |
tobiash | oh right | 17:38 |
tobiash | and I guess it's hard to mirror that | 17:38 |
clarkb | ya the issue is in telling the package to download from somewhere else | 17:38 |
clarkb | its been a while since I read the package source but I'm pretty sure it unconditionally does that download from its upstream | 17:38 |
mordred | tobiash: we could put the install in a retry loop | 17:42 |
mordred | (which would be stupid ugly - but might be a thing we want to consider?) | 17:42 |
mnaser | fwiw im pretty sure ansible-lint encourages retries for remote operations | 17:47 |
mnaser | thats what we do a lot of in OSA | 17:47 |
zbr | yes it does, but is not a genius, it will not detect network operations with shell :) | 17:48 |
mnaser | oh that's inside shell | 17:48 |
mnaser | i didnt dig into the context but yeah | 17:48 |
mnaser | we should probably just do that then imho. | 17:48 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 17:48 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 17:50 |
*** dpawlik has quit IRC | 17:51 | |
mnaser | ensure-yarn on it's own fails because it requires ensure-nodejs. should we document this or just add include_role ensure-nodejs ? | 17:51 |
mordred | mnaser: echo "APT::Acquire::Retries \"3\";" > /etc/apt/apt.conf.d/80-retries | 17:51 |
* mordred learned something today | 17:51 | |
mnaser | i'm in favour of #2 because it follows a similar pattern in things like ensure-kubernetes which does ensure-docker for example | 17:51 |
mordred | mnaser: #2 | 17:52 |
mordred | mnaser: I would expect ensure-yarn or ensure-k8s to do what they need to do to ensure that | 17:52 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on ensure-podman https://review.opendev.org/716578 | 17:52 |
mnaser | i'm terrified of putting a patch on top of the others which will make me carpet bomb zuul | 17:53 |
zbr | not sure if anyone cares but next version of ansible-lint will require ansible 2.8+ and py3+ | 17:53 |
clarkb | fwiw retrying network operations with shell is super easy. Use until | 17:53 |
clarkb | until install_cmd; do bump_incr; if [ incr > threshold ] ; then break ; fi ; done | 17:54 |
mordred | zbr: I do not think that will break anything here - but thanks! | 17:54 |
AJaeger | avass: I pushed https://review.opendev.org/717813 and https://review.opendev.org/717814 https://review.opendev.org/717815 for the install -> ensure rename. Will you do the rest, please? | 17:54 |
zbr | clarkb: i agree, rerunning entire shell block with until/retries is the nice and clean way of doing it. | 17:55 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: ensure-yarn: run ensure-nodejs before https://review.opendev.org/717817 | 17:55 |
mnaser | oh i didnt do anything | 17:56 |
mnaser | mordred: ^ as discussed, and made it self testing | 17:56 |
*** saneax has quit IRC | 17:56 | |
AJaeger | avass: what about preparing a change to remove the now obsolete install- roles as well? We should mark that as WIP. | 17:57 |
AJaeger | avass: so, your stack is approved - thanks! | 17:57 |
tristanC | zuul-maint: a few zuul-operator changes are waiting for reviews, in particular these two are causing rebase issue and it would be nice to have them merged to avoid that: https://review.opendev.org/717342 and https://review.opendev.org/707193 . Thanks in advance! | 17:59 |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-if-python to ensure-if-python for consistency https://review.opendev.org/716698 | 18:02 |
*** rlandy|lunch is now known as rlandy | 18:03 | |
avass | AJaeger: I can see what I can do :) | 18:06 |
*** lseki_ has joined #zuul | 18:07 | |
*** Miouge has quit IRC | 18:07 | |
*** lseki has quit IRC | 18:08 | |
*** lseki_ is now known as lseki | 18:08 | |
openstackgerrit | Tobias Henkel proposed zuul/zuul master: Retry nodeenv creation https://review.opendev.org/717820 | 18:08 |
zbr | mnaser: clarkb done rebase https://review.opendev.org/#/c/716578/ | 18:09 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby https://review.opendev.org/717371 | 18:09 |
*** Miouge has joined #zuul | 18:10 | |
tobiash | avass: in case you miss it because of the other +2, I commented on https://review.opendev.org/708871 | 18:14 |
zbr | does anyone know if become: "{{ item }}" works? i have the impression that it does not. | 18:15 |
AJaeger | thanks, avass - and if you need help, please ask! | 18:16 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: ensure-repositories: Add role https://review.opendev.org/717507 | 18:17 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: ensure-nodejs: refactor to use ensure-repositories https://review.opendev.org/717508 | 18:17 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: ensure-yarn: refactor to use ensure-repositories https://review.opendev.org/717509 | 18:17 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: ensure-yarn: run ensure-nodejs before https://review.opendev.org/717817 | 18:17 |
mnaser | AJaeger: lol. the _only_ role that wasn't self testing, had been broken :) | 18:18 |
openstackgerrit | Albin Vass proposed zuul/nodepool master: Use ensure-* roles https://review.opendev.org/717822 | 18:18 |
mnaser | i swapped debian/redhat accidentally in the vars file. found out in fixing ensure-yarn | 18:18 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 18:19 |
AJaeger | mnaser: ;) | 18:20 |
AJaeger | mnaser: can you add a self-test? | 18:20 |
mnaser | AJaeger: if i add self-test, it will break. therefore, the run ensure-nodejs fixes the role *and* adds self test (aka 717817) | 18:20 |
tristanC | tobiash: thank you for the review, it's really appreciated! | 18:20 |
tobiash | tristanC: no proble :) | 18:21 |
openstackgerrit | Albin Vass proposed zuul/zuul-base-jobs master: Remove ssh key in base cleanup run. https://review.opendev.org/708871 | 18:22 |
openstackgerrit | Albin Vass proposed zuul/zuul-base-jobs master: Remove ssh key in base cleanup run. https://review.opendev.org/708871 | 18:23 |
openstackgerrit | Albin Vass proposed zuul/zuul-base-jobs master: Remove ssh key in base cleanup run. https://review.opendev.org/717827 | 18:24 |
openstackgerrit | Tobias Henkel proposed zuul/zuul master: DNM: Test doc file comments https://review.opendev.org/716680 | 18:24 |
openstackgerrit | Albin Vass proposed zuul/zuul-base-jobs master: Remove ssh key in base-test cleanup run. https://review.opendev.org/708871 | 18:25 |
openstackgerrit | Albin Vass proposed zuul/zuul-base-jobs master: Remove ssh key in base cleanup run. https://review.opendev.org/717827 | 18:25 |
avass | tobiash: fixed :) | 18:26 |
tobiash | avass: thanks! | 18:26 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby https://review.opendev.org/717371 | 18:26 |
openstackgerrit | Merged zuul/zuul master: Make github file annotation levels configurable via zuul return https://review.opendev.org/711179 | 18:28 |
*** bhavikdbavishi has quit IRC | 18:28 | |
*** bhavikdbavishi has joined #zuul | 18:29 | |
openstackgerrit | Merged zuul/zuul master: Strip by default in tools/encrypt_secret https://review.opendev.org/714508 | 18:29 |
avass | AJaeger: should I push changes to all repos on review.opendev.org for s/install/ensure/ or just stick to opendev/openstack/zuul? | 18:32 |
avass | I mean, I guess everything needs to be updated if we're to remove the install- roles | 18:32 |
*** bhavikdbavishi has quit IRC | 18:34 | |
openstackgerrit | Mohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby https://review.opendev.org/717371 | 18:35 |
openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 18:38 |
mnaser | i always manage to find fun rabbit holes | 18:43 |
mnaser | we're installing nodejs 6.x by default in zuul-jobs from what it seems | 18:43 |
mnaser | which has been EOL'd since april 2019 | 18:44 |
mnaser | the most recent current active LTS is nodejs 10 | 18:45 |
mnaser | well, v10 and v12 are both in active LTS, v10 is the older, v12 is the newer | 18:45 |
mnaser | v13 seems like it should be entering LTS in a few days ago but not listed on the website | 18:46 |
mnaser | https://nodejs.org/en/about/releases/ | 18:46 |
mnaser | how do we go about updating our roles? :\ | 18:46 |
openstackgerrit | Merged zuul/zuul-operator master: Update dhall-kubernetes version to v4.0.0 https://review.opendev.org/717342 | 18:47 |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add job_volumes CR spec attribute https://review.opendev.org/706642 | 18:47 |
mnaser | i guess we can change the default behaviour and send a 2 week notice to ML | 18:51 |
tristanC | clarkb: maybe you'd be interested in reviewing https://review.opendev.org/706639 (adding job volumes to the operator spec) | 18:58 |
avass | AJaeger: there are still two changes for renaming the roles left: https://review.opendev.org/#/c/716689/ https://review.opendev.org/#/c/716752/ | 19:02 |
avass | mnaser: I think you accidentally changed the topic on the changes except for those two :) | 19:03 |
*** hasharAway is now known as hashar | 19:08 | |
openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Rename install-go to ensure-go for consistency https://review.opendev.org/716689 | 19:10 |
AJaeger | config-core, please review https://review.opendev.org/#/c/716752/ and https://review.opendev.org/#/c/716689/ | 19:17 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby https://review.opendev.org/717371 | 19:31 |
avass | AJaeger: I'll have to go, so I'll continue working on changing the install- role sto ensure- roles tomorrow | 19:34 |
AJaeger | avass: enjoy your evening - I'm signing off now as well... | 19:35 |
AJaeger | avass: and thanks! | 19:35 |
avass | no problem! bye :) | 19:35 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby https://review.opendev.org/717371 | 20:15 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby https://review.opendev.org/717371 | 20:25 |
mnaser | avass: oops, i did do that, my bad | 20:28 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby https://review.opendev.org/717371 | 20:39 |
*** Guest40054 has quit IRC | 20:42 | |
mnaser | corvus, mordred: does zuul-preview run in opendev somewhere right? | 20:52 |
mnaser | i ran into the issue with gatsby-ifying the website. https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_2c9/717371/11/check/gatsby-build/2c9d101/npm/html/ | 20:52 |
mnaser | thing are breaking because gatsby needs/depends on root urls | 20:52 |
corvus | mnaser: oh, there was a security problem with it and we shut it down; i need to fix it; shouldn't be hard, just didn't have any pressing need. maybe tomorrow? | 20:53 |
mnaser | corvus: oh cool, that wfm, i will start "refactoring" the gatsby jobs into zuul/zuul-jobs in the meantime given that i have it successfully building | 20:58 |
mnaser | it kinda sucks that it's not actually _technically_ successfully building though :\ | 20:58 |
mnaser | so i don't know if it's worth pushing that up there | 20:59 |
openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Update attributes to camelCase https://review.opendev.org/707193 | 20:59 |
*** jkt has quit IRC | 21:04 | |
*** jkt has joined #zuul | 21:04 | |
openstackgerrit | Mohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby https://review.opendev.org/717371 | 21:04 |
mordred | mnaser: http://localhost:8000/blog/zuul-is-cool/ is pretty awesome | 21:08 |
mordred | corvus: grab mnaser's change, run "yarn install ; yarn develop" and check out the magic | 21:09 |
openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: ensure-yarn: run ensure-nodejs before https://review.opendev.org/717817 | 21:10 |
mnaser | mordred: aha, it's actually based off our (vexxhost) blog so i kinda had to change it or otherwise it would have been articles from our site =P | 21:12 |
mordred | mnaser: :) | 21:12 |
mnaser | mordred: i was thinking of splitting off into two changes, one adding the blog later, but eh. | 21:12 |
mordred | mnaser: Zuul: "let vexxhost power your cloud journey!" | 21:12 |
mnaser | oh no did we leave that there lol | 21:12 |
mnaser | i git grep'd | 21:12 |
mordred | mnaser: no - I just thought that would be funny | 21:12 |
mnaser | mordred: aha, okay, fair nuff :p | 21:15 |
mnaser | mordred: btw given you're part of the masochist group of people who do some js-y stuff, what are your thoughts about earlier wrt the node version we have in zuul-jobs ? | 21:22 |
mnaser | tl;dr: we're installing node_version: 6 which has been EOL a year ago, wonder if we should bump it and how we go about that | 21:22 |
mordred | we should _totally_ bump the default | 21:25 |
mordred | probably should send a mailing list message warning that we're going to bump it in 2 weeks | 21:26 |
mordred | should we default it to 10 or to 12? | 21:26 |
*** hashar has quit IRC | 21:33 | |
fungi | i updated some of our stuff to nodejs 10.x because it's on ubuntu xenial still and didn't see any nodesource packages of 12.x for xenial | 21:37 |
fungi | but that may not be a concern here | 21:37 |
mordred | fungi: shouldn't be ... and one of the nice things here is that we don't need matching node versions on the running servers - only on build hosts | 21:38 |
mordred | so we can build on bionic with upstream node 13 - and still deploy on xenial | 21:39 |
mordred | would be more of an issue if people were running node as a server | 21:39 |
fungi | makes sense | 21:40 |
*** rlandy is now known as rlandy|afk | 21:50 | |
*** y2kenny has joined #zuul | 21:57 | |
openstackgerrit | James E. Blair proposed zuul/zuul-preview master: Don't proxy urls that don't appear in the map https://review.opendev.org/717870 | 22:02 |
corvus | mnaser: ^ that should get us moving again | 22:03 |
corvus | mordred: ^ | 22:03 |
*** rfolco has quit IRC | 22:03 | |
openstackgerrit | Merged zuul/zuul-jobs master: Rename install-go to ensure-go for consistency https://review.opendev.org/716689 | 22:03 |
mordred | corvus: seems reasonable | 22:04 |
fungi | oh, yep, i had forgotten about all that | 22:05 |
mnaser | mordred: I think we can do 13? It’s supppsed to have entered LTS a few days ago | 22:08 |
mnaser | I assume that’s just not updated yet: https://nodejs.org/en/about/releases/ | 22:08 |
mordred | mnaser: if we're going to bump a default, seems like a good target | 22:10 |
fungi | corvus: +2 on 717870 but also a question, just trying to make sure i grasp how that's working | 22:10 |
y2kenny | Does zuul scheduler automatically recover when ssh connection to gerrit timed out? | 22:15 |
mnaser | y2kenny: I believe it should | 22:18 |
y2kenny | ok | 22:18 |
fungi | we restart gerrit without restarting zuul, and it continues to work | 22:18 |
fungi | now depending on how it "times out" that behavior may vary | 22:19 |
clarkb | it won't discover events that happened during the outage though | 22:19 |
y2kenny | ok | 22:19 |
y2kenny | I missed some events and then turn out the connection timed out. | 22:19 |
y2kenny | I ended up restarting the scheduler | 22:20 |
fungi | certainly if the ssh session is disconnected via normal tcp/ip means (peer connection reset via rst packet, connection shutdown via fin/fin+ack or the like) then it works | 22:20 |
y2kenny | I think I saw an paramiko error | 22:20 |
y2kenny | or the time out came from paramiko error | 22:21 |
fungi | if the connection spontaneously goes away and zuul is never informed (maybe silently dropped by a firewall mid-session), then whether it finds out the connection is dead may depend on txp keepalive/dead peer detection settings in other places | 22:21 |
y2kenny | I didn't see any indication about reconnecting so I wasn't sure if it recovered or not. | 22:21 |
fungi | but those are more general challenges of maintaining long-lived tcp socket connections | 22:22 |
fungi | i don't think there's any ssh-level keepalive or dpd implemented, but that would depend on both paramiko(zuul side) and mina-sshd(gerrit side) i think | 22:23 |
y2kenny | I see | 22:25 |
*** hashar has joined #zuul | 22:26 | |
fungi | y2kenny: aha! well-trodden ground: https://zuul-ci.org/docs/zuul/reference/drivers/gerrit.html#attr-%3Cgerrit%20connection%3E.keepalive | 22:26 |
y2kenny | cool! | 22:27 |
fungi | by default it does ssh keepalive with an ssh protocol zero-content chunk every 60 seconds, looks like | 22:27 |
*** sshnaidm is now known as sshnaidm|afk | 22:28 | |
y2kenny | I will keep an eye on it and see if this happens again | 22:29 |
fungi | cool | 22:31 |
y2kenny | for the upload-log, is it correct to say that it is an ansible job being run on the log server to rsync from the executor to the log server? | 22:31 |
y2kenny | I got the upload-log working last week but for some reason it stopped working (I got ssh: connect to host <log server> port 22: Connection refused) | 22:31 |
y2kenny | even though the other jobs (like add-fileserver and set zuul-log-path) seems to be working... | 22:32 |
y2kenny | I can see the directories like the "zuul_log_path" are created... the failure seems to be on the rsync bit. | 22:33 |
y2kenny | and I can rsync from my workstation to the log server using the ssh key that's encrypted... um... | 22:34 |
fungi | we stopped using it in favor of log storage, but i thought it ran rsync from the executor to push content to the logserver | 22:35 |
fungi | er, stopped using it in favor of object storage (in opendev's zuul deployment) | 22:35 |
y2kenny | yea... this is my intermediate setup | 22:36 |
y2kenny | something simple and quick and it was working last week... I just need to traceback and figure out what I changed | 22:36 |
fungi | so can rsync on the executor connect to the logserver? | 22:36 |
y2kenny | I can ssh but I haven't tried rsync... I should give that a try | 22:37 |
fungi | connection refused implies it's either ssh not listening on 22/tcp on the logserver or maybe some firewall rules not allowing it | 22:37 |
fungi | well, rsync seems to be trying to use ssh protocol there, so ssh seems like a reasonable test | 22:37 |
fungi | if rsync is getting a connection refused on 22/tcp then presumably so would ssh (or anything trying to connect to that port for that matter) | 22:38 |
openstackgerrit | Merged zuul/zuul-preview master: Don't proxy urls that don't appear in the map https://review.opendev.org/717870 | 23:07 |
fungi | corvus: thanks for the answer on 717870, that confirmed it's working the way i thought it was at least. i agree both solutions are roughly equivalent | 23:11 |
fungi | corvus: also i think the example here basically matches what you wound up doing, so seems like as fine a choice as they offer: http://httpd.apache.org/docs/2.4/rewrite/rewritemap.html | 23:13 |
fungi | you merely took the additional step to make sure the default couldn't accidentally be shadowed by a local file | 23:14 |
fungi | which seems like a reasonable precaution | 23:15 |
*** tosky has quit IRC | 23:28 | |
*** hashar has quit IRC | 23:32 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!