Wednesday, 2020-11-04

*** rlandy has quit IRC00:11
*** wuchunyang has joined #zuul00:14
*** wuchunyang has quit IRC00:18
*** smyers has quit IRC00:21
*** smyers has joined #zuul00:45
*** holser has quit IRC01:02
*** Goneri has quit IRC01:23
*** armstrongs has joined #zuul01:26
*** armstrongs has quit IRC01:36
*** hamalq has quit IRC01:38
openstackgerritIan Wienand proposed zuul/zuul-jobs master: merge-output-to-logs: convert to untrusted executor safe code  https://review.opendev.org/75832502:44
*** bhavikdbavishi has joined #zuul02:54
*** bhavikdbavishi1 has joined #zuul02:57
*** yolanda has quit IRC02:57
*** bhavikdbavishi has quit IRC02:58
*** bhavikdbavishi1 is now known as bhavikdbavishi02:58
openstackgerritMerged zuul/zuul-jobs master: More E208 fixes  https://review.opendev.org/76109003:14
*** zenkuro has quit IRC04:28
*** bhavikdbavishi has quit IRC04:28
*** bhavikdbavishi has joined #zuul04:29
openstackgerritMerged zuul/zuul master: Make protected attribute clearer to the end user  https://review.opendev.org/75922604:34
*** evrardjp has quit IRC05:33
*** evrardjp has joined #zuul05:33
*** wuchunyang has joined #zuul05:45
openstackgerritSimon Westphahl proposed zuul/zuul master: Implementation of Zookeeper backed event queues  https://review.opendev.org/76117005:51
openstackgerritSimon Westphahl proposed zuul/zuul master: Implementation of Zookeeper event watcher  https://review.opendev.org/76117105:51
openstackgerritSimon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed trigger event queues  https://review.opendev.org/76117205:51
*** bhavikdbavishi1 has joined #zuul06:25
*** vishalmanchanda has joined #zuul06:26
*** bhavikdbavishi has quit IRC06:27
*** bhavikdbavishi1 is now known as bhavikdbavishi06:27
*** saneax has joined #zuul06:38
*** bhavikdbavishi has quit IRC07:11
*** mach1na has joined #zuul07:18
*** mach1na has quit IRC07:18
*** mach1na has joined #zuul07:22
tobiashianw: responded on https://review.opendev.org/75809207:25
*** bhavikdbavishi has joined #zuul07:43
*** mach1na has quit IRC07:48
*** mach1na has joined #zuul07:52
openstackgerritFelix Edel proposed zuul/zuul master: Configure json-server as mock API for development  https://review.opendev.org/76093307:53
openstackgerritzbr proposed zuul/zuul-jobs master: Enable progressive mode with ansible-lint  https://review.opendev.org/76069108:02
*** jcapitao has joined #zuul08:05
*** sshnaidm|afk is now known as sshnaidm|rover08:08
openstackgerritzbr proposed zuul/zuul-jobs master: More E208  https://review.opendev.org/76129308:21
*** bhavikdbavishi1 has joined #zuul08:22
*** bhavikdbavishi has quit IRC08:23
*** bhavikdbavishi1 is now known as bhavikdbavishi08:23
openstackgerritFelix Edel proposed zuul/zuul master: PF4: Rework of log viewer page  https://review.opendev.org/75114008:25
openstackgerritFelix Edel proposed zuul/zuul master: UI: Avoid empty state being shown before data is fetched  https://review.opendev.org/75853008:25
openstackgerritFelix Edel proposed zuul/zuul master: UI: Remove refresh button from build and buildset page  https://review.opendev.org/75853408:25
openstackgerritFelix Edel proposed zuul/zuul master: Configure json-server as mock API for development  https://review.opendev.org/76093308:25
openstackgerritzbr proposed zuul/zuul-jobs master: More E208 (22)  https://review.opendev.org/76129408:32
*** bhavikdbavishi has quit IRC08:34
*** bhavikdbavishi has joined #zuul08:34
openstackgerritTobias Henkel proposed zuul/zuul master: Remove an unneeded api call when creating check_runs  https://review.opendev.org/75205608:42
openstackgerritTobias Henkel proposed zuul/zuul master: Save superfluous api requests in check run reporting  https://review.opendev.org/75207908:42
*** tosky has joined #zuul08:42
*** hashar has joined #zuul08:54
*** jpena|off is now known as jpena08:56
openstackgerritzbr proposed zuul/zuul-jobs master: More E208 (final)  https://review.opendev.org/76129708:58
ianwtobiash: thanks; i should probably have noted more clearly that it just didn't seem to work for me; i will give it another go09:00
ianwlike all the api pages were 40409:00
tobiashianw: are you referring to 758092?09:01
ianwyep09:01
tobiashianw: that's weird, the test cases work and we have it in production as well. I'd be interesting in the root cause09:02
*** rpittau|afk is now known as rpittau09:05
ianwtobiash: maybe it's my yarn/npm mix up; i couldn't see anything from the 3001 server but didn't look too much.  tomorrow i'll try clearing things and see if can start again09:06
tobiashk09:06
openstackgerritSimon Westphahl proposed zuul/zuul master: Add and fix fields in driver trigger event models  https://review.opendev.org/76116609:14
openstackgerritSimon Westphahl proposed zuul/zuul master: Allow (de-)serialization of trigger events  https://review.opendev.org/76116709:14
openstackgerritSimon Westphahl proposed zuul/zuul master: Interface to get a driver's trigger event class  https://review.opendev.org/76116809:14
openstackgerritSimon Westphahl proposed zuul/zuul master: Clear list of Zookeeper connections after tests  https://review.opendev.org/76116909:14
openstackgerritSimon Westphahl proposed zuul/zuul master: Implementation of Zookeeper backed event queues  https://review.opendev.org/76117009:14
openstackgerritSimon Westphahl proposed zuul/zuul master: Implementation of Zookeeper event watcher  https://review.opendev.org/76117109:14
openstackgerritSimon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed trigger event queues  https://review.opendev.org/76117209:14
*** bhavikdbavishi has quit IRC09:33
*** holser has joined #zuul09:52
*** ironfoot has quit IRC09:56
*** ironfoot has joined #zuul09:58
*** hashar has quit IRC10:10
*** wuchunyang has quit IRC10:18
openstackgerritSimon Westphahl proposed zuul/zuul master: Driver event ingestion  https://review.opendev.org/71729911:06
openstackgerritSimon Westphahl proposed zuul/zuul master: Move management and result events to model  https://review.opendev.org/76116311:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Allow (de-)serialization of management events  https://review.opendev.org/76116411:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Allow (de-)serialization of result events  https://review.opendev.org/76116511:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Add and fix fields in driver trigger event models  https://review.opendev.org/76116611:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Allow (de-)serialization of trigger events  https://review.opendev.org/76116711:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Interface to get a driver's trigger event class  https://review.opendev.org/76116811:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Clear list of Zookeeper connections after tests  https://review.opendev.org/76116911:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Implementation of Zookeeper backed event queues  https://review.opendev.org/76117011:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Implementation of Zookeeper event watcher  https://review.opendev.org/76117111:08
openstackgerritSimon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed trigger event queues  https://review.opendev.org/76117211:08
*** bhavikdbavishi has joined #zuul12:11
*** mach1na has quit IRC12:11
*** jcapitao is now known as jcapitao_lunch12:15
openstackgerritAshley Bullock proposed zuul/zuul master: Add initial bitbucket cloud driver using webhooks  https://review.opendev.org/75900312:19
*** AshBullock has joined #zuul12:25
AshBullockHey all, wanted to catch up on the bitbucketcloud driver I've been working on, one annoying thing I can't find a good way around is https://jira.atlassian.com/browse/BCLOUD-5814, which prevents forks from working currently, the pr ref spec has been a feature request for 8 years :(12:28
*** jpena is now known as jpena|lunch12:32
*** rlandy has joined #zuul12:33
*** hashar has joined #zuul12:34
*** armstrongs has joined #zuul12:52
*** mach1na has joined #zuul12:59
*** rfolco has joined #zuul13:07
*** jcapitao_lunch is now known as jcapitao13:13
*** armstrongs has quit IRC13:16
*** holser has quit IRC13:18
tobiashAshBullock: you mean you cannot fetch the pr head from the target repo if the pr was created from a fork?13:22
AshBullocktobiash yeah, i've tried a few things out, but i'm finding the forks commit references just aren't present on the target repo, reading through that feature request it seems everyone trying to integrate with bitbucketcloud is seeing similar issues13:26
AshBullockI'm sure there is some way around it i'm not seeing, but currently it limits to same repo branches13:26
*** Goneri has joined #zuul13:37
*** bhavikdbavishi has quit IRC13:39
*** jpena|lunch is now known as jpena13:40
*** zenkuro has joined #zuul13:50
*** saneax has quit IRC13:52
*** saneax has joined #zuul13:59
sean-k-mooneyAshBullock: i guess you could work around it byt having the bitbucket driver add the PR source repo as a remote and pull the branch form there14:07
sean-k-mooneybut really it is not something you should have to do14:07
*** saneax has quit IRC14:08
*** mach1na has quit IRC14:11
*** zenkuro has quit IRC14:12
*** zenkuro has joined #zuul14:13
*** mach1na has joined #zuul14:14
*** holser has joined #zuul14:25
fungiif zuul can already identify the source repo for the pr, couldn't the merger just fetch the commit id for it? doesn't actually require a named ref nor configuring an additional remote (you can fetch from a repository url, doesn't have to be configured as a remote first)14:28
corvusi think fetch by sha is new; you'd want to confirm that works14:29
corvus"new" is relative of course :)14:29
fungigood point, it works on *my* git, but maybe not *old* git14:31
corvusand who knows what bbc is running14:31
fungioh, you mean depends on the git server version14:32
corvusyes -- i think that's the case.  but am i remembering that wrong and it's only client?14:32
fungii'm testing various combinations right now14:33
fungigit 1.9.1 client (ubuntu 14.04) can't do it, claims "error: no such remote ref"14:34
fungi2.7.4 (ubuntu 16.04) also can't, same error14:38
fungi2.17.1 (ubuntu 18.04) works14:38
tobiashthe official docs about fetching a pr is this: https://confluence.atlassian.com/bbkb/how-to-locally-fetch-and-checkout-a-pull-request-724402529.html14:38
fungibut that's specifically when testing against opendev's gerrit 2.13 (whatever jgit rev that is)14:38
tobiashso tldr: git fetch <merge request url>14:39
tobiashwith the source branch14:39
tobiashnot sha14:39
tobiashbut I wonder what happens if the forked repo has different permissions than the original one14:39
tobiash(if that's possible at all)14:39
corvusAshBullock: ^ fyi14:40
openstackgerritSimon Westphahl proposed zuul/zuul master: Allow (de-)serialization of trigger events  https://review.opendev.org/76116714:40
openstackgerritSimon Westphahl proposed zuul/zuul master: Interface to get a driver's trigger event class  https://review.opendev.org/76116814:40
openstackgerritSimon Westphahl proposed zuul/zuul master: Clear list of Zookeeper connections after tests  https://review.opendev.org/76116914:40
openstackgerritSimon Westphahl proposed zuul/zuul master: Implementation of Zookeeper backed event queues  https://review.opendev.org/76117014:40
openstackgerritSimon Westphahl proposed zuul/zuul master: Implementation of Zookeeper event watcher  https://review.opendev.org/76117114:40
openstackgerritSimon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed trigger event queues  https://review.opendev.org/76117214:40
tobiashproblem is that this makes it hard to integrate into zuul since the merger's architecture currently assumes a single remote url14:42
fungialso tested against opendev's gitea instance and i can similarly fetch by commit id (sha1 hash)14:42
corvustobiash: true, but it's a tractable problem; we could pass a new url all the way through if necessary14:43
tobiashyes14:43
tobiashif there is a new url the merger needs to do two fetches14:43
tobiashjust not as easy as implementing a driver ;)14:43
fungiwould probably need to find a bb repo with an open pr and try closing the pr's sha i suppose, i'll see if i can find one quickly14:43
fungier, s/closing/fetching/14:44
fungiyeah, it works14:49
fungiAshBullock: example... http://paste.openstack.org/show/799694/14:50
fungiso maybe not ideal, and as tobiash says it'll likely require a new merger feature to plumb the pr source repo name through14:50
tobiashfungi: can you fetch the same sha from the original repo as well?14:50
fungioh, good question, i'll try that. i know it works on github only because of the way they designed their backend/forking solution14:51
tobiashI guess not since that ticket is not mentioning such a workaround14:51
fungiso i wouldn't expect that to be a universal behavior14:51
fungierror: Server does not allow request for unadvertised object 26b7b5ec07f4b8f6f1f2dfb18ea1685dfe7fdfae14:52
fungiso that's a no, unfortunately14:52
tobiashk so double fetch is the way to go14:52
fungibut like i said, that only works on github because they combine all objects for forks into the same space (doesn't even need a pr, you can fork a repo on github, push some new commits to it, then reference that commit id from the original repo)14:53
*** bhavikdbavishi has joined #zuul14:53
fungithat's been abused to amusing effect during the youtube-dl dmca takedown, to push copies of the youtube-dl source to a fork of github's dmca takedown repo and then publish references to it in the original. even if the fork is deleted the objects persist (supposedly indefinitely unless gh staff get a tipoff and manually delete said objects)14:55
funginot quite the "security hole" folks keep making it out to be, but still rather humorous14:56
AshBullockThanks for the help, i've hit the same issue tobiash mentions, the sha is not present on the original repo unfortunately, so it looks like I'll need to look into the bespoke merger method14:58
*** bhavikdbavishi has quit IRC15:00
AshBullockon another note, I'm adding test cases for the driver, I'm following similar examples from the base.py mock examples, it would be great to catch up with someone on this as I just want to make sure I'm following the correct format15:00
*** bhavikdbavishi has joined #zuul15:01
AshBullockalso I've reached out to Maciej, who i've seen has made some great progress on the server implementation, just to check if we can consolidate anywhere15:01
*** bhavikdbavishi has quit IRC15:29
*** bhavikdbavishi has joined #zuul15:30
openstackgerritMatthieu Huin proposed zuul/zuul-website master: Add link to zuul-client documentation  https://review.opendev.org/75131215:54
AshBullockcorvus tobiash fungi in your view would fork support with the additional merger method be a requirement before this gets put in?16:10
fungiAshBullock: i think as long as some workflow is supported we could start with what works and then try to add pr workflow support. the gitlab driver had similar feature parity gaps when we merged the initial support commits for it16:12
AshBullockor could we footnote that branch support is the only option for now, I'm just conscious of adding unneeded complexity to the code base, but seeing that feature request's open time i'm not confident it will be implemented any time soon, it's quite baffling16:13
AshBullockok great, well, I'll look into the merger bits and see how much work it would be16:14
corvusAshBullock: yeah, i don't think we need to block on it, but i think it's important and we should plan on adding it soon.16:16
AshBullocka nice evening project! yeah, otherwise i'll add some bits to the docs on the issue16:17
*** Goneri has quit IRC16:18
openstackgerritDaniel Pawlik proposed zuul/zuul master: Improve Elasticsearch reporter doc and driver, changed index name  https://review.opendev.org/76144116:18
*** Goneri has joined #zuul16:21
*** mach1na has quit IRC16:26
*** hashar has quit IRC16:28
*** hashar has joined #zuul16:28
AshBullockfor testing, i'm working through test cases now, is the desired approach to add mocks for connection, pr etc to tests/base.py?16:30
openstackgerritClark Boylan proposed zuul/nodepool master: Have nodepool scan as many ssh host keys as possible  https://review.opendev.org/76122916:30
clarkbcorvus: ^ that is updates based on test job logs. I'm trying to make the logging cleaner there16:30
tobiashclarkb: do you have an estimation of the performance impact that has?16:50
tobiashlooks like that does six keyscans per node?16:50
openstackgerritMerged zuul/zuul master: tools: use python3 for encrypt_secret  https://review.opendev.org/76026616:51
clarkbtobiash: on from the test logs, https://zuul.opendev.org/t/zuul/build/6fe013f39a9d4f0ba3abab796162ed9c/log/nodepool/nodepool-launcher.log#955, it looks to be about a second and a half spent key scanning16:51
tobiashk16:52
clarkbcompared to the wait time for ssh to listen as well as general spin loop for server to boot I expect the impact to be small16:52
mhusince 760266 got merged, how about adding the encrypt subcommand to zuul-client? https://review.opendev.org/#/c/750709/ and https://review.opendev.org/#/c/754103/16:53
pabelangertristanC: thanks again, that got me the data we needed.17:14
pabelangertalking with Goneri, I think we are going to see about updating it to use REST API17:14
*** tosky has quit IRC17:15
pabelangertobiash: clarkb: left comment too17:17
clarkbthe logs are much cleaner with the latest patchset in the openstack functional job and shows it is adding rsa, ecdsa, and ed25519 keys17:22
clarkbpabelanger: note I think there are at least 3 different ecdsa key types17:22
pabelangerright17:23
pabelangeron centos, it is default 1 for bit size17:23
pabelangerhaven't changed others17:23
pabelangerchecked817:23
pabelangertristanC: also, is the status-page report that is generated for SF live some place?17:24
clarkbthe risk there is if you pick one ecdsa key size that your client decides is not valid later (because ansible uses openssh not paramiko) you'll break17:24
clarkbit really does seem more flexible to grab all the valid keys and add them then let ssh validate those that it sees17:24
pabelangeragree17:25
*** jcapitao has quit IRC17:25
fungithat way the answer to supporting additional host key types is just to make sure they get generated at boot17:28
pabelangeror deleted on boot17:28
*** rpittau is now known as rpittau|afk17:35
*** vishalmanchanda has quit IRC17:38
tristanCpabelanger: nice, sf-config runs status-page daily at fqdn/status, here is the one for sf-project.io: https://softwarefactory-project.io/status/17:38
pabelangergreat, thank you17:39
pabelangertristanC: I'd love to see if we could agree to some sort of basic report page for zuul dashboard17:39
pabelangerbut know that might be hard, since a lot of business logic17:39
tristanCpabelanger: ideally we would define a /metric endpoint, then we could share prometheus and grafana config that would work out of the box17:40
tristanCi think that would be the most ergonomic things to do17:40
*** hamalq has joined #zuul17:46
*** jpena is now known as jpena|off18:04
*** sugaar has quit IRC18:13
*** reiterative has quit IRC18:14
*** mugsie has quit IRC18:18
*** mugsie has joined #zuul18:21
*** bhavikdbavishi has quit IRC18:32
*** holser has quit IRC19:32
*** sduthil has quit IRC20:22
*** Goneri has quit IRC20:23
*** tosky has joined #zuul20:35
*** Goneri has joined #zuul20:37
*** Goneri has quit IRC20:46
sean-k-mooneyis there a reason the zuul.conf options for the componets are grouped in https://zuul-ci.org/docs/zuul/discussion/components.html instead of https://zuul-ci.org/docs/zuul/reference/admin.html with the driver and connection options21:14
sean-k-mooneywhen i was redeploying my third party ci this split signifcantly increase the barrier to entry as its really un intuitive for the config for the zuul execucutor/schduler/web app to live under discussion not under reference21:16
*** Goneri has joined #zuul21:26
*** Goneri has quit IRC21:51
*** hashar has quit IRC22:01
*** holser has joined #zuul22:08
*** rfolco has quit IRC22:30
*** rlandy is now known as rlandy|bbl23:07
*** AshBullock has quit IRC23:40
*** armstrongs has joined #zuul23:51

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!