| *** jamesmcarthur has quit IRC | 00:14 | |
| *** jamesmcarthur has joined #zuul | 00:15 | |
| *** jamesmcarthur has quit IRC | 00:16 | |
| *** jamesmcarthur has joined #zuul | 00:16 | |
| *** jamesmcarthur has quit IRC | 00:17 | |
| *** jamesmcarthur has joined #zuul | 00:19 | |
| fungi | okay, gave it an hour just to be on the safe side. seeing no additional feedback or suggested edits, i'm sending now. thanks! | 00:31 |
|---|---|---|
| *** jamesmcarthur has joined #zuul | 00:35 | |
| *** jamesmcarthur has quit IRC | 00:52 | |
| *** ianychoi has quit IRC | 01:16 | |
| *** ianychoi has joined #zuul | 01:16 | |
| *** jamesmcarthur has joined #zuul | 01:32 | |
| *** saneax has joined #zuul | 01:37 | |
| *** jamesmcarthur has quit IRC | 01:55 | |
| *** bhavikdbavishi has joined #zuul | 02:06 | |
| *** bhavikdbavishi1 has joined #zuul | 02:09 | |
| *** bhavikdbavishi has quit IRC | 02:11 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 02:11 | |
| *** bhavikdbavishi has quit IRC | 02:13 | |
| *** bhavikdbavishi has joined #zuul | 02:25 | |
| *** bhavikdbavishi1 has joined #zuul | 02:30 | |
| *** jamesmcarthur has joined #zuul | 02:31 | |
| *** ianychoi has quit IRC | 02:31 | |
| *** bhavikdbavishi has quit IRC | 02:32 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 02:32 | |
| *** ianychoi has joined #zuul | 02:33 | |
| *** jamesmcarthur has quit IRC | 02:38 | |
| *** spsurya has joined #zuul | 02:50 | |
| *** ianychoi has quit IRC | 02:51 | |
| *** ianychoi has joined #zuul | 02:52 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Stub out dry run in the uploader https://review.opendev.org/592929 | 02:58 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Create a download script https://review.opendev.org/592341 | 02:58 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Add a unicode file https://review.opendev.org/592853 | 02:58 |
| *** rfolco has quit IRC | 03:15 | |
| *** saneax has quit IRC | 03:30 | |
| *** bolg has joined #zuul | 05:11 | |
| *** ianychoi has quit IRC | 05:13 | |
| *** ianychoi has joined #zuul | 05:14 | |
| *** ianychoi has quit IRC | 05:26 | |
| *** ianychoi has joined #zuul | 05:28 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Create a download script https://review.opendev.org/592341 | 05:32 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Add a unicode file https://review.opendev.org/592853 | 05:32 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: testing autogenerated download script https://review.opendev.org/689930 | 05:32 |
| *** bolg has quit IRC | 06:06 | |
| *** themroc has joined #zuul | 06:08 | |
| *** sshnaidm_ has joined #zuul | 06:15 | |
| *** sshnaidm|pto has quit IRC | 06:15 | |
| *** igordc has joined #zuul | 06:34 | |
| *** pcaruana has joined #zuul | 06:47 | |
| *** igordc has quit IRC | 06:48 | |
| *** saneax has joined #zuul | 07:04 | |
| *** bolg has joined #zuul | 07:11 | |
| *** tosky has joined #zuul | 07:12 | |
| *** sshnaidm_ has quit IRC | 07:14 | |
| *** hashar has joined #zuul | 07:18 | |
| *** sshnaidm has joined #zuul | 07:27 | |
| *** jangutter has joined #zuul | 07:52 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Zuul Web: add /api/user/authorizations endpoint https://review.opendev.org/641099 | 08:10 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: gerrit: add support for report only connection https://review.opendev.org/568216 | 08:32 |
| fbo | tristanC: corvus do you think this patch ^ could be replaced by the new" stream_events connection setting ? | 08:34 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: gerrit: add support for report only connection https://review.opendev.org/568216 | 08:36 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: trigger: add job filter event https://review.opendev.org/639905 | 09:30 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: webtrigger: add initial driver and event https://review.opendev.org/555153 | 09:33 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: webtrigger: add web route and rpclistener https://review.opendev.org/554839 | 09:35 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: web: add build button to trigger job https://review.opendev.org/635716 | 09:36 |
| *** mauved has quit IRC | 09:49 | |
| *** mauved has joined #zuul | 09:50 | |
| *** phildawson has quit IRC | 10:05 | |
| *** pcaruana has quit IRC | 10:16 | |
| *** pcaruana has joined #zuul | 10:21 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Create a download script https://review.opendev.org/592341 | 10:38 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: test bulk download script change https://review.opendev.org/689930 | 10:38 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Add a unicode file https://review.opendev.org/592853 | 10:38 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: test bulk download script change https://review.opendev.org/689930 | 10:43 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Create a download script https://review.opendev.org/592341 | 10:43 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Add a unicode file https://review.opendev.org/592853 | 10:43 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Add a unicode file https://review.opendev.org/592853 | 11:26 |
| *** ianychoi has quit IRC | 11:37 | |
| *** ianychoi has joined #zuul | 11:40 | |
| *** ianychoi has quit IRC | 12:00 | |
| *** yolanda has quit IRC | 12:01 | |
| *** ianychoi has joined #zuul | 12:03 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: WIP: Allow ensure-tox to upgrade tox version https://review.opendev.org/676464 | 12:05 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: change default calling as a module https://review.opendev.org/690057 | 12:12 |
| *** yolanda has joined #zuul | 12:13 | |
| *** jamesmcarthur has joined #zuul | 12:18 | |
| *** jamesmcarthur has quit IRC | 12:30 | |
| *** ianychoi has quit IRC | 12:39 | |
| *** ianychoi has joined #zuul | 12:41 | |
| *** ianychoi has quit IRC | 12:54 | |
| *** rfolco has joined #zuul | 12:55 | |
| *** ianychoi has joined #zuul | 12:57 | |
| jangutter | This is an odd question, but does anyone use zuul and git submodules together (I know it's a bad idea, but legacy, y'know). It would improve our workflow immensely if we can get the merger to clone with git --recursive as an option. | 13:34 |
| jangutter | I just want to check if someone's hit a similar problem before I start hacking on it. | 13:35 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: change default calling as a module https://review.opendev.org/690057 | 13:38 |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Take advantage of fetch-output https://review.opendev.org/689473 | 13:39 |
| corvus | jangutter: http://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2019-10-16.log.html#t2019-10-16T22:34:55 | 13:47 |
| jangutter | Do you guys have some sort of non-causal way of discussing my questions the week before? | 13:48 |
| corvus | jangutter: mailing list, and if it's a more complex subject we need to get agreement on, a spec | 13:49 |
| corvus | jangutter: do you think mordred's suggestion would cover your use-case? | 13:50 |
| jangutter | corvus: I'm checking - we're specifically _not_ considering the CI aspect (can of worms), but it would be nice if the submodules could be checked out by zuul before the roles start. | 13:52 |
| corvus | fbo: yes, i think you can use the stream events setting instead | 13:56 |
| *** ianychoi has quit IRC | 13:58 | |
| *** ianychoi has joined #zuul | 14:00 | |
| jangutter | corvus: modred's suggestion would be pretty much next-level supercool. Gleaning dep info from submodules is not even something I had considered. CI-ing submodules that have not merged I kind-of considered as impossible. | 14:02 |
| fungi | i wonder if i'm the only one who got jangutter's temporal paradox joke | 14:03 |
| *** rfolco is now known as rfolco|rover | 14:03 | |
| jangutter | fungi: would be more impressive if you caught it 20 minutes ago, but I'm not complaining. | 14:03 |
| fungi | bwahahah | 14:04 |
| jangutter | OK, just to double-check my understanding: | 14:11 |
| jangutter | 1) open up a review on the master repo that depends-on a review of the submodule. | 14:11 |
| jangutter | 2) Zuul scans the master repo and figures out that the submodule repo needs to have an checkout override to the submodule dir. | 14:11 |
| jangutter | 3) Zuul creates a merge commit on the master repo, as if the submodule review has merged. | 14:11 |
| jangutter | (I didn't consider the CI implications, I was thinking of adding a flag that clones the work repos with --recursive when you opt-in) | 14:12 |
| jangutter | Step 3 is the domain of dragons. | 14:13 |
| corvus | 1 and 2 are correct -- i'm not certain that mordred was thinking of implementing 3 in exactly that way, but rather achieving the same effect by zuul having prepared the submodule repo in the normal way of a required-project and then performing the equivalent of a git init. (having said that, i think item 3 may be an equally valid and achievable way to implement that) | 14:20 |
| corvus | (er, 'git submodule init') | 14:20 |
| *** igordc has joined #zuul | 14:23 | |
| corvus | clarkb, fungi, mordred: https://github.com/kislyuk/rehash/issues/4 is an interesting read | 14:24 |
| fungi | hah | 14:25 |
| fungi | "i can work on fixing this once our ci system has python 3.8 support" | 14:25 |
| jangutter | See, the acausal logic is spreading. | 14:26 |
| fungi | we could be facing a total causality collapse, yes | 14:26 |
| fungi | lots of me-too comments on https://github.com/actions/setup-python/issues/30 | 14:27 |
| fungi | alan clark just complimented zuul and opendev for being able to keep up with the volume and complexity of testing required to support the openstack train release | 14:28 |
| fungi | (on the osf board of directors call) | 14:28 |
| corvus | aww thanks! | 14:29 |
| clarkb | if only there was some way to collaboratively build a CI system that met the needs of its users :) | 14:29 |
| clarkb | that said I thought github actions ran with user specified containers? you can probably supply a python-slim container and get 3.8 that way? | 14:29 |
| corvus | here's the file: https://github.com/kislyuk/rehash/blob/master/.github/workflows/pythonpackage.yml | 14:29 |
| *** michael-beaver has joined #zuul | 14:30 | |
| corvus | so maybe it could use something like "runs-on: python:3.8-slim" and drop the actions/setup-python step? | 14:30 |
| corvus | (i have no idea, i'm just making that up) | 14:31 |
| corvus | clarkb: apparently they are virtual machines: https://help.github.com/en/github/automating-your-workflow-with-github-actions/virtual-environments-for-github-actions | 14:33 |
| clarkb | TIL | 14:34 |
| corvus | apparently "use vms for all your ci tests" is not as passe as some people have suggested. | 14:34 |
| jangutter | decided lack of CentOS :-/ | 14:37 |
| *** ianychoi has quit IRC | 14:38 | |
| corvus | it looks like they have special "actions" for docker containers, so there still may be a not-too-complex way to use python:slim | 14:39 |
| *** ianychoi has joined #zuul | 14:40 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Test buildset registry with k8s and docker https://review.opendev.org/689280 | 14:42 |
| corvus | clarkb, fungi: want to push https://review.opendev.org/689880 through? | 14:42 |
| clarkb | done | 14:44 |
| openstackgerrit | Merged zuul/project-config master: Add gerrit image build job to gerrit checks plugin https://review.opendev.org/689880 | 14:44 |
| fungi | heh, that updated to merged just as i pulled it up | 14:45 |
| fungi | the noop is *fast* ;) | 14:45 |
| mnaser | doing nothing is fast D: | 14:48 |
| jangutter | you _quite_ sure the noop didn't complete before the +W? | 14:50 |
| *** jamesmcarthur has joined #zuul | 14:53 | |
| *** pcaruana has quit IRC | 14:58 | |
| Shrews | FYI, probably afk for the rest of the day. Having car issues while running an errand this morning. | 15:07 |
| corvus | Shrews: good luck! | 15:09 |
| corvus | clarkb, fungi: i don't understand this: https://zuul.opendev.org/t/zuul/build/2b5c1df748f74d4db75f5ddde4334955 | 15:14 |
| corvus | that's for a recheck of a zuul-jobs change which has otherwise run many times | 15:14 |
| clarkb | corvus: is it possible pip/setuptools updated and started making that an error? | 15:15 |
| corvus | that tried to install 3.10.2; the last time that job ran, it apparently installed 3.2.0 https://zuul.opendev.org/t/zuul/build/c30cb8d6c05b4abf93bf103702897ae0/log/tox/py27-1.log | 15:15 |
| fungi | looking | 15:16 |
| clarkb | https://opendev.org/zuul/zuul-jobs/src/branch/master/test-requirements.txt#L6 is the dependency | 15:16 |
| corvus | maybe? or maybe something happened with mirroring, and 3.2.0 didn't have that restriction? | 15:16 |
| fungi | yeah, looks like python 2.7 pip tried to install zuul | 15:17 |
| fungi | and so is complaining that zuul says it only works with python 3.5 and later | 15:17 |
| corvus | but apparently that usually works? | 15:17 |
| fungi | or something changed | 15:17 |
| clarkb | possible that different pip version would find the newest zuul that works with python2 and install that and that has changed? | 15:18 |
| corvus | the same job ran 3 minutes later and passed | 15:18 |
| fungi | huh, have a link so we can compare the logs? | 15:18 |
| fungi | i can't imagine the py27 job would have been successfully installing zuul v3 | 15:19 |
| corvus | pass: https://zuul.opendev.org/t/zuul/build/0b37898faa8445b19293da4a9122e3ae fail: https://zuul.opendev.org/t/zuul/build/2b5c1df748f74d4db75f5ddde4334955 | 15:19 |
| fungi | thanks | 15:19 |
| corvus | different changes, but they're in the same series (the one i'm rechecking over and over); none touch anything related to unit tests | 15:19 |
| *** saneax has quit IRC | 15:20 | |
| clarkb | the same version of pip is used in both cases | 15:20 |
| *** yolanda__ has joined #zuul | 15:21 | |
| clarkb | I think pypi will provide valid versions for the requesting python version | 15:21 |
| fungi | https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_0b3/689280/10/check/tox-py27/0b37898/tox/py27-1.log | 15:21 |
| fungi | you can see there the py27 tox log shows it installing http://mirror.ord.rax.opendev.org/pypifiles/packages/39/54/09c7e809b5b216c99c165f8390837c5d0814350026beaaac126eb73d81d5/zuul-3.2.0.tar.gz | 15:21 |
| fungi | Successfully installed [...] zuul-3.2.0 | 15:22 |
| clarkb | ya and 3.2.0 is the newest version that doesn't specify requires >= 3.5 on pypi | 15:22 |
| fungi | bizarre | 15:22 |
| clarkb | I think it is possible that this is due to pypi | 15:23 |
| clarkb | pip is relying on it to give it the valid version list? | 15:23 |
| fungi | yeah, and the failing job tries to install http://mirror.sjc1.vexxhost.openstack.org/pypifiles/packages/e7/48/a083a6639aed280151d3c43e690d5a7dd445f729278f5ba08f87f5a65300/zuul-3.10.2.tar.gz | 15:23 |
| clarkb | which is not the latest version either | 15:23 |
| corvus | i wonder if the cdn cached the wrong results | 15:24 |
| *** yolanda has quit IRC | 15:24 | |
| *** mattw4 has joined #zuul | 15:26 | |
| corvus | we going with "<shrug> pypi. see if it happens again?" | 15:27 |
| clarkb | ya I think so as I'm 95% sure that it is pypi that filters the python versions for pip | 15:28 |
| corvus | k. i'll keep an eye | 15:28 |
| *** sgw has quit IRC | 15:29 | |
| webknjaz | nope | 15:29 |
| webknjaz | it's Pip that does version filtering | 15:29 |
| webknjaz | if it's new enough | 15:29 |
| fungi | webknjaz: but the filtering is going to be based on the response it gets from pypi | 15:30 |
| webknjaz | PyPI just show whatever the maintainer uploaded in dist meta | 15:30 |
| clarkb | ah ok | 15:30 |
| clarkb | webknjaz: the metadata shows 3.10.2 requires python >= 3.5 any idea why pip might try to install that anyway on python3? | 15:30 |
| clarkb | er on python2 | 15:30 |
| webknjaz | @fungi: Pip looks up the page with a simple indes, like https://pypi.org/simple/ansible/ | 15:31 |
| clarkb | the version of pip used was the same as a job that properly filtered it out | 15:31 |
| webknjaz | Then, it look at the names of dists listed there | 15:31 |
| fungi | webknjaz: we've seen truncated responses from the pypi fastly cdn endpoints in the past | 15:31 |
| fungi | so can't rule out incomplete responses | 15:31 |
| webknjaz | based on the name pattern, Pip can filter out things | 15:31 |
| webknjaz | but yes, it probably looks up some metadata from the API (so that it won't have to download dists first to figure it out) | 15:32 |
| webknjaz | > any idea why pip might try to install that anyway on python3 | 15:32 |
| webknjaz | yes, it's because of the outdated Pip version | 15:32 |
| webknjaz | It should be v8+ or v10+, don't remember exactly | 15:33 |
| clarkb | wait so it does get that info from pypi? | 15:33 |
| clarkb | also the version of pip was 19.2.3 | 15:34 |
| webknjaz | I think so | 15:34 |
| clarkb | could still be pypi then | 15:34 |
| webknjaz | Also, you can do `-vvv` to make Pip more verbose, maybe it'll reveal the reason why it chose that dist | 15:34 |
| clarkb | I think we go with that original "see if it happens again" | 15:34 |
| webknjaz | https://www.irccloud.com/pastebin/bgrmon54/ | 15:36 |
| webknjaz | > zuul-3.10.2.tar.gz | 15:36 |
| webknjaz | why does it hit sdist? | 15:36 |
| clarkb | because there is no wheel for that version | 15:36 |
| clarkb | *for that python + zuul version | 15:36 |
| webknjaz | I'm not sure sdist can ship this meta | 15:36 |
| clarkb | right which is why I was fairly certain it comes from pypi | 15:37 |
| clarkb | because without first downloading all of the sdists and unpacking them the info wouldn't be availalb | 15:37 |
| webknjaz | OTOH I downloaded that sdist and it has python-requires | 15:39 |
| webknjaz | ➜ grep \\-Python zuul-3.10.2/zuul.egg-info/PKG-INFO | 15:39 |
| webknjaz | Requires-Python: >=3.5 | 15:39 |
| corvus | view-source:http://mirror.ord.rax.opendev.org/pypi/simple/zuul/ | 15:40 |
| corvus | that does have the python version req in it | 15:40 |
| clarkb | aha | 15:40 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: change default calling as a module https://review.opendev.org/690057 | 15:40 |
| clarkb | it does come from pypi | 15:40 |
| webknjaz | wait, where? | 15:41 |
| corvus | view-source:http://mirror.sjc1.vexxhost.openstack.org/pypi/simple/zuul/ | 15:41 |
| webknjaz | stop | 15:41 |
| webknjaz | I was wrong | 15:41 |
| corvus | that does not | 15:41 |
| webknjaz | no, it doesn't have version info there | 15:41 |
| webknjaz | that's a simple-index page, with links | 15:41 |
| webknjaz | Pip downloads it over HTTP and finds all matching filenames | 15:41 |
| corvus | data-requires-python=">=3.5" | 15:42 |
| clarkb | webknjaz: check the html source | 15:42 |
| clarkb | corvus: in that case I think that would be a cdn issue? we are proxying caching in each of those mirror nodes so are dependent on the data provided by pypi's cdn | 15:42 |
| webknjaz | I'm not sure that Pip uses that attr | 15:42 |
| webknjaz | and the filename doesn't have Python branch specifier in it | 15:42 |
| corvus | they're even sorted differently | 15:43 |
| webknjaz | what's sorted differently? | 15:44 |
| corvus | the two simple index pages from the different mirror locations | 15:45 |
| corvus | (vexxhost-sjc1 vs rax-ord) | 15:45 |
| webknjaz | looks like his page has invalid relative URLs: http://mirror.ord.rax.opendev.org/pypi/simple/ | 15:48 |
| webknjaz | `/simple/PKG_NAME` instead of `/pypi/simple/PKG_NAME` | 15:49 |
| corvus | last time i looked, pip ignored that and only used the package names from the index page | 15:50 |
| *** bhavikdbavishi has quit IRC | 15:50 | |
| *** bhavikdbavishi has joined #zuul | 15:51 | |
| *** ianychoi has quit IRC | 15:52 | |
| webknjaz | Well, Pip should open the package page to see the list of dists for that package | 15:52 |
| webknjaz | https://www.irccloud.com/pastebin/BCcnSjmr/ | 15:52 |
| webknjaz | It goes to the usual PyPI CDN on my laptop | 15:53 |
| *** ianychoi has joined #zuul | 15:54 | |
| webknjaz | https://www.irccloud.com/pastebin/WluCSLWM/ | 15:54 |
| webknjaz | And if I disable the default index, it just doesn't find anything ^ | 15:54 |
| webknjaz | if you add `-vvv` it'll explain what it skips pages of every dist listed there | 15:58 |
| webknjaz | https://www.irccloud.com/pastebin/akPRViDW/ | 15:59 |
| webknjaz | If you check any link like `Skipping link http://mirror.ord.rax.opendev.org/simple/mast/ (from http://mirror.ord.rax.opendev.org/pypi/simple/); not a file`, it returns 404 | 15:59 |
| clarkb | that is orthogonal to the issue at hand though right? I mena it is something we may want to address in the proxy cache system but unrelated to the switch since we've got the difference of behavior through this proxy setup | 16:00 |
| webknjaz | what's different? | 16:00 |
| clarkb | corvus pointed out the difference is order of entries and the lack of the data-requires-python tags | 16:01 |
| webknjaz | I don't think it matters much | 16:01 |
| clarkb | I mean it is clearly downloading the packages from from that mirror successfully | 16:01 |
| clarkb | it is also deciding that it cannot install what it has downloaded | 16:02 |
| clarkb | (which implies to me the problem you point out isorthogonal to the problem corvus has run into) | 16:02 |
| webknjaz | what do you mean by deciding it cannot install? | 16:02 |
| clarkb | webknjaz: https://zuul.opendev.org/t/zuul/build/2b5c1df748f74d4db75f5ddde4334955 ERROR: Package 'zuul' requires a different Python: 2.7.15 not in '>=3.5' | 16:03 |
| corvus | yes, in our usage of pip, the index page is hardly used (it's really only used to correct the case of package names) | 16:03 |
| *** themroc has quit IRC | 16:03 | |
| clarkb | and apparently the metadata tags for that info are present in one of our pypi caches and not another | 16:04 |
| clarkb | implying to me that there is a difference in cdn setup? | 16:04 |
| *** bolg has quit IRC | 16:04 | |
| clarkb | the one that fails is the one without the data-requires-python | 16:05 |
| clarkb | the one that works has that information | 16:05 |
| webknjaz | okay, it seems like Pip looks at that since v9 | 16:06 |
| webknjaz | https://github.com/pypa/pip/blob/937315905670d25dc7354a832d81ea3e4afacef1/NEWS.rst#L874 | 16:06 |
| webknjaz | https://github.com/pypa/pip/blob/0f095af4ba7ac1af2c27024fa539af4c3a45d002/src/pip/_internal/index/collector.py#L232 | 16:07 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: WIP: Allow ensure-tox to upgrade tox version https://review.opendev.org/676464 | 16:07 |
| openstackgerrit | James E. Blair proposed zuul/project-config master: Use gerrit master job with no files matcher https://review.opendev.org/690121 | 16:08 |
| clarkb | corvus: I believe the ttl on those files is short ~10 minutes | 16:08 |
| clarkb | corvus: if it persists we should confirm that then escalate to pypi | 16:08 |
| corvus | clarkb: ack | 16:08 |
| corvus | jackpot! got a v6 registry node: https://zuul.opendev.org/t/zuul/build/f0587812badc40aea9a807bf126e0ef7 | 16:10 |
| *** igordc has quit IRC | 16:20 | |
| *** yolanda__ has quit IRC | 16:38 | |
| *** pcaruana has joined #zuul | 16:41 | |
| *** pcaruana has quit IRC | 16:52 | |
| *** bhavikdbavishi has quit IRC | 17:15 | |
| *** hashar has quit IRC | 17:15 | |
| openstackgerrit | Merged zuul/project-config master: Use gerrit master job with no files matcher https://review.opendev.org/690121 | 17:22 |
| *** ianychoi has quit IRC | 17:39 | |
| *** ianychoi has joined #zuul | 17:42 | |
| *** ianychoi has quit IRC | 17:55 | |
| *** ianychoi has joined #zuul | 17:58 | |
| mnaser | hey -- just need a second pair of eyes to double check | 18:00 |
| mnaser | https://zuul-ci.org/docs/zuul/admin/tenants.html seems to mention that tenant.config-projects and tenant.untrusted-projects are a thing, but the layout is tenant.source.<connection>.{config,untrusted}-projects | 18:00 |
| mnaser | that seems like a documentation bug (i can fix, but just wanted a second pair of eyes) | 18:01 |
| *** jamesmcarthur has quit IRC | 18:01 | |
| corvus | mnaser: yeah, i think the text in https://zuul-ci.org/docs/zuul/admin/tenants.html#attr-tenant.source explains, but it's not obvious, and that all probably predates our use of metasyntactic variables, so i think an update to do that would be more clear | 18:03 |
| mnaser | ok ill push up a small patch now | 18:03 |
| clarkb | indenting another level might help too to show that is an attribute of the source? though that might make things hard to read if it gets too squished | 18:04 |
| openstackgerrit | James E. Blair proposed zuul/project-config master: Run opendev-buildset-registry when building gerrit https://review.opendev.org/690146 | 18:07 |
| corvus | fungi, clarkb: ^ quick +3 on that? | 18:07 |
| mnaser | clarkb: yeah that was what i was thinknig but i was worried it would start to get a lil more confusing | 18:07 |
| mnaser | btw, there seems to be a little disagreement in teh docs too | 18:08 |
| mnaser | https://zuul-ci.org/docs/zuul/user/config.html "Pipeline configuration items may only appear in config-projects." | 18:08 |
| mnaser | yet.. https://zuul-ci.org/docs/zuul/admin/tenants.html#attr-tenant.untrusted-projects.%3Cproject%3E.include | 18:08 |
| mnaser | seems to imply that you can include/exclude pipelines there? | 18:08 |
| corvus | heh. it's "technically" correct. you can put the word "pipeline" in the include list there, and it will not be a syntax error, however it will be ignored. | 18:09 |
| corvus | maybe the easiest fix for that is a "note: pipelines are ignored in untrusted projects" | 18:10 |
| corvus | or some such | 18:10 |
| openstackgerrit | Merged zuul/project-config master: Run opendev-buildset-registry when building gerrit https://review.opendev.org/690146 | 18:10 |
| clarkb | or have docs for include in config-projects and specify different lists? | 18:10 |
| clarkb | I think the issue there is it is documenting all valid include options for config-projects and untrusted-projects | 18:11 |
| corvus | i think it's legitimate to note that it's not a syntax error (it's important when you start to talk about default values, etc) | 18:11 |
| corvus | like, you can make a single list of stuff that includes pipelines, and throw config and unstrusted projects in that list, and that is valid syntax but pipelines will still be ignored in unstrusted projects. | 18:11 |
| corvus | why do i keep saying unstrusted? that's weird. | 18:12 |
| corvus | untrusted. untrusted. untrusted. | 18:12 |
| *** ianychoi has quit IRC | 18:12 | |
| *** sgw has joined #zuul | 18:13 | |
| *** igordc has joined #zuul | 18:14 | |
| corvus | we've scheduled jobs on https://gerrit-review.googlesource.com/c/plugins/checks/+/241293 at https://zuul.opendev.org/t/zuul/status | 18:14 |
| *** ianychoi has joined #zuul | 18:16 | |
| Shrews | well that was certainly an expensive errand | 18:16 |
| corvus | Shrews: how's your new car? | 18:16 |
| Shrews | pfft, new? i wish | 18:16 |
| Shrews | parts of it are new now | 18:17 |
| *** ianychoi has quit IRC | 18:26 | |
| *** ianychoi has joined #zuul | 18:30 | |
| *** spsurya has quit IRC | 18:30 | |
| *** ianychoi has quit IRC | 18:38 | |
| *** ianychoi has joined #zuul | 18:42 | |
| mnaser | http://zuul.opendev.org/api/tenant/opendev/build/108c06c64b5745b581ad999156df78a3 | 18:43 |
| mnaser | is there a reasoning behind why node_name is empty? | 18:43 |
| *** hwangbo has joined #zuul | 18:45 | |
| hwangbo | Hello, can I define different projects and queues for different branches in the same untrusted gerrit repo? | 18:46 |
| mnaser | hwangbo: you could technically have a .zuul.yaml file inside every branch containing the config for that branch itself -- i think that would get you what you want? | 18:46 |
| *** hashar has joined #zuul | 18:47 | |
| *** jamesmcarthur has joined #zuul | 18:47 | |
| *** igordc has quit IRC | 18:47 | |
| hwangbo | We have a zuul.yaml file in each branch, but they still ended up in the same dependent pipeline queue, even though different ones were specified | 18:47 |
| hwangbo | So I was wondering if that ability was something supported, to have different dependent queues for different branches on the same repo | 18:49 |
| *** jamesmcarthur_ has joined #zuul | 18:50 | |
| *** michael-beaver has quit IRC | 18:50 | |
| clarkb | hwangbo: no, and that is somewhat intentional to enable upgrade testing | 18:51 |
| hwangbo | all right, thank you for confirming! | 18:53 |
| *** jamesmcarthur has quit IRC | 18:54 | |
| fungi | hwangbo: specifically, it's intended to make sure that if a job involves more than one branch of your repo, changes to one branch are tested with any changes queued for other branches | 18:55 |
| *** igordc has joined #zuul | 18:56 | |
| hwangbo | clarkb fungi: so if I have different queues defined for different branches, is it basically first branch/change in the pipeline determines what queue it belongs in? | 18:57 |
| fungi | hwangbo: all configuration from all branches is read by zuul and combined | 18:57 |
| fungi | but yes, i think in that case one of them would override the queue value set in the others | 18:58 |
| fungi | o | 18:58 |
| fungi | i'm not sure which branch would take precedence in that case | 18:58 |
| corvus | the first one zuul encounters: https://zuul-ci.org/docs/zuul/user/config.html#attr-project.%3Cpipeline%3E.queue | 18:58 |
| clarkb | hwangbo: what is the goal with splitting the queues? | 18:59 |
| clarkb | (it might help to understand the use case to know if there is some other approach that could be taken) | 18:59 |
| corvus | so put it in a config-project, or in the master branch of the project itself. | 18:59 |
| fungi | but the idea with not separating branches into different queues is so that if you have a 1.x branch and a 2.x branch and a job that tests upgrading from 1.x to 2.x, then it makes sure that changes for 1.x and 2.x which are queued at the same time get tested together so you don't merge a change for one of them which doesn't work with the changes for the other in the job | 18:59 |
| hwangbo | clarkb: teams working on different branches didn't want the dependent queue length/size to affect each other | 18:59 |
| hwangbo | so if there are 20 changes in queue for branch A, they didn't want a change for branch B to have to wait for all 20 to finish | 19:01 |
| corvus | well, they don't. zuul runs those in parallel. | 19:01 |
| corvus | that's kind of it's main thing. | 19:01 |
| fungi | well, they do have to wait for all 20 ahead to finish before they can merge the change, but they don't have to wait for all 20 to merge before jobs for the change can start being tested | 19:02 |
| openstackgerrit | James E. Blair proposed zuul/project-config master: Remove gerrit checks plugin third-party-check pipeline https://review.opendev.org/690307 | 19:02 |
| hwangbo | Also, if our pipeline window is small (for whatever reason, flaky testing, bad test nodes), it's also affected in those scenarios | 19:03 |
| corvus | i'm going to self-approve that ^ | 19:03 |
| *** pcaruana has joined #zuul | 19:04 | |
| *** ianychoi has quit IRC | 19:04 | |
| corvus | that's certainly true :/ | 19:04 |
| clarkb | though you can increase the window size or disable it entirely iirc | 19:04 |
| clarkb | (typically the best solution there is to stop attempting to merge changes that can't possibly merge, but I understand it is often difficult to know that ahead of time when approving things) | 19:05 |
| *** jamesmcarthur has joined #zuul | 19:05 | |
| corvus | in openstack we've generally accepted that being able to test upgrades is worth the cost. i suppose if you had a piece of software with no upgrade testing, then branch/queue separation might make sense. | 19:06 |
| *** jamesmca_ has joined #zuul | 19:06 | |
| openstackgerrit | Merged zuul/project-config master: Remove gerrit checks plugin third-party-check pipeline https://review.opendev.org/690307 | 19:07 |
| fungi | but yeah, at least at the moment, zuul assumes there could be some interaction or dependency relationships between all branches of a project and errs on the side of correctness | 19:07 |
| hwangbo | That's definitely fair, thanks for the clarification | 19:07 |
| *** ianychoi has joined #zuul | 19:08 | |
| fungi | right now you can use depends-on freely between changes for different branches of the same project and get the expected behavior. if branches could be configured to not share the same queue as other branches for the same project, you'd need additional configuration constructs to be able to identify when there are branch relationships to make things like depends-on work correctly | 19:08 |
| *** jamesmcarthur_ has quit IRC | 19:08 | |
| *** jamesmcarthur has quit IRC | 19:09 | |
| *** jamesmcarthur has joined #zuul | 19:10 | |
| *** hwangbo has quit IRC | 19:12 | |
| *** jamesmcarthur has quit IRC | 19:12 | |
| *** jamesmca_ has quit IRC | 19:13 | |
| *** hwangbo has joined #zuul | 19:13 | |
| *** jamesmcarthur has joined #zuul | 19:15 | |
| *** ianychoi has quit IRC | 19:15 | |
| hwangbo | another question then, since I have all these different zuul.yaml files across different branches, is there a way to specify, maybe on a repo level, "use the zuul.yaml from this specified branch and ignore the others"? | 19:15 |
| hwangbo | one that hopefully doesn't require me to open a gerrit review for each branch :) | 19:15 |
| fungi | if it's a config-project then only the master branch is considered | 19:19 |
| fungi | however you also don't get speculative execution of proposed configuration changes in those | 19:19 |
| corvus | hwangbo: not at the moment, though we have discussed that. but there may be other options for specific issues... what sort of thing are you thinking you might need to change on all the branches? | 19:19 |
| *** pcaruana has quit IRC | 19:19 | |
| hwangbo | just remove the different queues, so they don't interfere with each other. I'm worried that if it's first-come defines the queue, shared queue items in different projects will not be processed correctly | 19:20 |
| *** ianychoi has joined #zuul | 19:24 | |
| clarkb | hwangbo: one way to address that is the define the queues in a config project without branches | 19:25 |
| clarkb | that will be loaded before the in repo configs of untrusted projects | 19:25 |
| *** jamesmcarthur has quit IRC | 19:27 | |
| hwangbo | clarkb: that might work, I'll look into it. thanks! | 19:27 |
| corvus | hwangbo: yeah, clarkb's suggestion is the best -- but also, it is determistic. ie, it's based on the order zuul loads configs, so if it's defined in the project itself, it is always certain that zuul will use the value in the master branch, because it always starts there. | 19:30 |
| corvus | (so you can be sure that values in other branches will be ignored -- it's safe to leave them there) | 19:31 |
| *** jamesmcarthur has joined #zuul | 19:32 | |
| *** ianychoi has quit IRC | 19:39 | |
| *** hwangbo has quit IRC | 19:43 | |
| *** jamesmcarthur has quit IRC | 19:46 | |
| openstackgerrit | James E. Blair proposed zuul/project-config master: Make third-party-check quieter https://review.opendev.org/690405 | 20:24 |
| corvus | clarkb, fungi, mordred: ^ that should take care of the chatty messages and fix the urls that we send | 20:24 |
| openstackgerrit | James E. Blair proposed zuul/project-config master: Revert "Remove gerrit checks plugin third-party-check pipeline" https://review.opendev.org/690407 | 20:28 |
| *** hwangbo has joined #zuul | 20:51 | |
| openstackgerrit | Merged zuul/project-config master: Make third-party-check quieter https://review.opendev.org/690405 | 21:01 |
| corvus | clarkb: something isn't adding up for me with this registry ipv6 thing. the issue is when we're in limestone (all ipv6). the job makes a complete replica of the environment (fake executor, intermediate registry, and builder+buildset registry). we set up a socat process on the executor so that we can skopeo copy to the buildset registry (via the socat port on localhost). that isn't actually able to | 21:25 |
| corvus | contact the buildset registry in the test job. i don't actually see how this is any different than production, and am wondering if copying to a production buildset registry is actually working. but i'm sure we thought we fixed this (we put a lot of effort into the socat stuff). am i missing something? | 21:25 |
| corvus | (i suspect the immediate problem is actually that docker isn't setting up the necessary iptables rules for v6; but i'm hesitant to just dive into that without knowing whether or why this works in prod) | 21:27 |
| corvus | oh wait i found it! | 21:30 |
| corvus | we open ports 5000:5001 in ip6tables in the opendev buildset-registry job pre-playbook | 21:31 |
| corvus | so we need to do the same thing in the test job | 21:31 |
| *** armstrongs has joined #zuul | 21:40 | |
| *** mattw4 has quit IRC | 21:40 | |
| clarkb | corvus: catching up after lunch and a bike ride and ya the ipv6 ports was a problem in production for a while too | 21:40 |
| clarkb | that was the lsat piece of that puzzle, once we put that into prod it worked reliably | 21:40 |
| *** mattw4 has joined #zuul | 21:41 | |
| corvus | cool, i guess we just lucked out with this role's testing so far | 21:41 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Use zuul-registry as buildset registry https://review.opendev.org/689238 | 21:42 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Make the buildset registry port configurable https://review.opendev.org/689240 | 21:42 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Test buildset registry with k8s and docker https://review.opendev.org/689280 | 21:42 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Open iptables ports in registry test job https://review.opendev.org/690425 | 21:42 |
| corvus | clarkb, mordred: i think we're ready for https://review.opendev.org/690407 unless anyone thinks of something else to check before trying again | 21:43 |
| mordred | corvus: it took me several reads of the above to grok it - the testing of testing things can be mindbending | 21:44 |
| mordred | corvus: I also think we're ready for it | 21:44 |
| openstackgerrit | Merged zuul/project-config master: Revert "Remove gerrit checks plugin third-party-check pipeline" https://review.opendev.org/690407 | 21:44 |
| corvus | mordred: yeah -- though being abel to say "this should be just like production" really helps | 21:45 |
| corvus | that's what reminded me to go look to see if the base job did anything else | 21:45 |
| mordred | corvus: SPEAKING OF ..., | 21:45 |
| mordred | actually, let me move this to -infra | 21:45 |
| *** armstrongs has quit IRC | 21:46 | |
| corvus | mordred: round 2 of gerrit testing is happening in https://zuul.opendev.org/t/zuul/status now | 21:49 |
| *** hashar has quit IRC | 21:51 | |
| mordred | corvus: sweet! | 22:04 |
| mordred | corvus: btw - at some point it might be nice to tenant-scope the cookie we use for project filters | 22:04 |
| corvus | ++ | 22:05 |
| mordred | (I was just confused that I didn't see anything, but it was because I had system-config still in the list from openstack tenant) | 22:05 |
| mordred | https://gerrit-review.googlesource.com/c/plugins/checks/+/241293/ <-- | 22:06 |
| mordred | you can see the check is in the running state | 22:06 |
| mordred | clarkb: ^^ | 22:06 |
| clarkb | looks like it failed and the view log buildset link worked | 22:07 |
| clarkb | One thing I like about our existing js hacks is that you see the table by default | 22:07 |
| corvus | hrm, still left a message though :/ | 22:07 |
| clarkb | maybe we can suggest the checks api render by default per a user set option or something | 22:07 |
| mordred | clarkb: there is a thing coming called "subchecks" | 22:09 |
| corvus | yeah, i'm not sure why you would only want to see either files or checks. | 22:09 |
| mordred | oh - nevermind - I misunderstood | 22:09 |
| mordred | and yeah - the checks are pretty hidden for now | 22:09 |
| corvus | but if you click on the "1 of 1 checks failed" it does open the table for you | 22:09 |
| mordred | corvus: what is the "re-run" link supposed to accomplish? | 22:10 |
| corvus | mordred: it removes the check result, causing the next poll to indicate that it should run a check | 22:11 |
| mordred | ah - neat | 22:11 |
| corvus | mordred: it should not work for your regular account | 22:11 |
| mordred | should I try? | 22:11 |
| corvus | mordred: go for it :) | 22:11 |
| mordred | it didn't work - and also didn't give me an error :) | 22:12 |
| corvus | sounds about right :) | 22:12 |
| corvus | i'm assuming that's a TODO | 22:12 |
| corvus | it does work under the opendev-zuul account, so i'm logged in with that so i can click it to trigger this stuff | 22:12 |
| corvus | (though i could also "zuul enqueue") | 22:12 |
| corvus | i think we may need further code changes to zuul to suppress the job reporting in comments | 22:14 |
| mordred | corvus: I noticed OpenDev Zuul responded to David | 22:14 |
| corvus | yeah, i got the wrong tab :( | 22:14 |
| mordred | corvus: :) | 22:14 |
| mordred | corvus: I was thinking zuul had achieved sentience | 22:15 |
| openstackgerrit | James E. Blair proposed zuul/project-config master: Revert "Revert "Remove gerrit checks plugin third-party-check pipeline"" https://review.opendev.org/690430 | 22:16 |
| corvus | self-approving ^ | 22:16 |
| clarkb | "I am corvus" | 22:16 |
| clarkb | mordred: ^ | 22:16 |
| openstackgerrit | Merged zuul/project-config master: Revert "Revert "Remove gerrit checks plugin third-party-check pipeline"" https://review.opendev.org/690430 | 22:16 |
| mnaser | https://review.opendev.org/#/c/688446/ | 22:16 |
| mordred | clarkb: if zuul told me it was corvus, how would I know? | 22:16 |
| mnaser | does anyone mind kicking +W-ing this? | 22:16 |
| mnaser | wow words are hard | 22:17 |
| corvus | done | 22:17 |
| mnaser | thanks! | 22:18 |
| mnaser | https://zuul.opendev.org/t/opendev/build/360d08f189824abdae6ae324cc75b109 | 22:34 |
| mnaser | hmm, it blew up | 22:34 |
| *** saneax has joined #zuul | 22:49 | |
| *** tosky has quit IRC | 22:56 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Add a unicode file https://review.opendev.org/592853 | 23:01 |
| *** tjgresha_ has joined #zuul | 23:46 | |
| *** tjgresha_ has quit IRC | 23:51 | |
| *** tjgresha has joined #zuul | 23:52 | |
| *** tjgresha has quit IRC | 23:53 | |
| *** igordc has quit IRC | 23:55 | |
| *** mattw4 has quit IRC | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!