| *** jamesmcarthur has joined #zuul | 00:18 | |
| *** jamesmcarthur has quit IRC | 00:32 | |
| *** jamesmcarthur has joined #zuul | 00:34 | |
| *** jamesmcarthur has quit IRC | 00:44 | |
| *** jamesmcarthur has joined #zuul | 00:51 | |
| *** jamesmcarthur has quit IRC | 01:01 | |
| *** jamesmcarthur has joined #zuul | 01:02 | |
| *** holser has quit IRC | 01:03 | |
| *** jamesmcarthur has quit IRC | 01:04 | |
| *** jamesmcarthur has joined #zuul | 01:04 | |
| *** jamesmcarthur has quit IRC | 01:11 | |
| *** jamesmcarthur has joined #zuul | 01:13 | |
| *** jamesmcarthur has quit IRC | 01:16 | |
| *** jamesmcarthur has joined #zuul | 01:18 | |
| *** jamesmcarthur has quit IRC | 01:20 | |
| *** jamesmcarthur has joined #zuul | 01:30 | |
| *** jamesmcarthur has quit IRC | 01:33 | |
| *** ikhan has quit IRC | 01:37 | |
| *** jamesmcarthur has joined #zuul | 01:53 | |
| *** jamesmcarthur has quit IRC | 02:00 | |
| *** iurygregory has joined #zuul | 02:07 | |
| *** jamesmcarthur has joined #zuul | 02:15 | |
| *** jamesmcarthur has quit IRC | 02:19 | |
| *** jamesmcarthur has joined #zuul | 02:22 | |
| *** jamesmcarthur has quit IRC | 02:25 | |
| *** jamesmcarthur has joined #zuul | 02:40 | |
| *** jamesmcarthur has quit IRC | 03:06 | |
| *** sanjayu_ has joined #zuul | 03:13 | |
| *** ikhan has joined #zuul | 03:21 | |
| *** cloudnull has quit IRC | 03:38 | |
| *** vishalmanchanda has joined #zuul | 04:00 | |
| *** bhavikdbavishi has joined #zuul | 04:23 | |
| *** bhavikdbavishi1 has joined #zuul | 04:30 | |
| *** ykarel has joined #zuul | 04:31 | |
| *** bhavikdbavishi has quit IRC | 04:32 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 04:32 | |
| *** jamesmcarthur has joined #zuul | 05:10 | |
| *** jamesmcarthur has quit IRC | 05:14 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** bhavikdbavishi has quit IRC | 06:01 | |
| *** sanjayu_ has quit IRC | 06:02 | |
| *** saneax has joined #zuul | 06:06 | |
| *** bhavikdbavishi has joined #zuul | 06:24 | |
| *** bhavikdbavishi1 has joined #zuul | 06:27 | |
| *** bhavikdbavishi has quit IRC | 06:28 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 06:28 | |
| *** saneax has quit IRC | 06:37 | |
| *** saneax has joined #zuul | 06:45 | |
| *** cloudnull has joined #zuul | 06:53 | |
| *** piotrowskim has joined #zuul | 07:08 | |
| *** bhavikdbavishi has quit IRC | 07:25 | |
| *** jcapitao has joined #zuul | 07:45 | |
| *** jpena|off is now known as jpena | 07:50 | |
| *** jfoufas1 has joined #zuul | 07:52 | |
| *** hashar has joined #zuul | 08:01 | |
| *** bhavikdbavishi has joined #zuul | 08:07 | |
| *** bhavikdbavishi1 has joined #zuul | 08:10 | |
| *** bhavikdbavishi1 has quit IRC | 08:10 | |
| *** bhavikdbavishi has quit IRC | 08:10 | |
| *** bhavikdbavishi has joined #zuul | 08:13 | |
| *** ykarel is now known as ykarel|lunch | 08:30 | |
| *** rpittau|afk is now known as rpittau | 08:33 | |
| msuszko | I've run into performance issues on zuul-scheduler few weeks ago, and resolved it thanks to tobiash asynchronous reporting change (691253). | 08:39 |
|---|---|---|
| *** ykarel|lunch has quit IRC | 08:39 | |
| msuszko | There was also lots of thime spent in chardet module, so I monkey-patched requests to use cchardet | 08:40 |
| msuszko | s/thime/time/ | 08:41 |
| msuszko | Yesterday deployed updated version, but not bothered with cchardet this time, and zuul-schedulere hogs on one core with less than 20% of traffic | 08:44 |
| tobiash | msuszko: are you using gerrit or github? | 08:45 |
| msuszko | So it is chardet, three orders of magnitude slower than cchardet, and it is used in gerrit driver. | 08:45 |
| tobiash | I wonder if chardet can be completely avoided there | 08:46 |
| tobiash | I guess gerrit is utf-8 only so chardet may not be needed at all there if there is a possibility to skip its use in requests | 08:46 |
| msuszko | Do you have a clue, how to "politely" make requests use it, without monkey-patching requests/compat.py? | 08:47 |
| tobiash | not really | 08:47 |
| msuszko | oh, that is true | 08:48 |
| *** yoctozepto has quit IRC | 08:48 | |
| tobiash | msuszko: you could try to use r.raw here: https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/gerrit/gerritconnection.py#L630 | 08:51 |
| tobiash | maybe that avoids the chardet usage in requests | 08:51 |
| msuszko | there is proper Content-Type header with charset in Gerrits reponse, I wonder what this chardet is for | 08:52 |
| *** bhavikdbavishi has quit IRC | 08:54 | |
| tobiash | docs of r.text says if Response.encoding is None encoding will be guessed using chardet | 08:54 |
| tobiash | so maybe the response object doesn't know the encoding | 08:54 |
| tobiash | https://github.com/psf/requests/blob/master/requests/models.py#L839 | 08:55 |
| tobiash | this docstring also includes a hint how to avoid guessing | 08:56 |
| tobiash | I guess that's easier than monky patching chardet | 08:56 |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 08:57 | |
| tobiash | further we access r.text four times in that method and it seams that that property doesn't cache the result so the conversion is done several times | 08:58 |
| tobiash | so I think we should fetch the text once and work on that then | 08:58 |
| *** zenkuro has joined #zuul | 09:18 | |
| *** vishalmanchanda has quit IRC | 09:18 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Call r.text only once https://review.opendev.org/c/zuul/zuul/+/770071 | 09:24 |
| tobiash | msuszko: you can try this ^ | 09:24 |
| tobiash | corvus: this might be useful for opendev as well | 09:25 |
| openstackgerrit | Merged zuul/zuul master: Handle WIP Gerrit status https://review.opendev.org/c/zuul/zuul/+/769436 | 09:38 |
| msuszko | i'm trying to pinpoint this with pptop | 09:52 |
| msuszko | i'm not sure if this is r.text | 10:06 |
| msuszko | in profiler it is mostly chardet.jpcntx.SJISContextAnalysis.get_order and chardet.jpcntx.EUCJPContextAnalysis.get_order, and it seems like trying to figure out encoding od nontext data | 10:07 |
| tobiash | can you see the calling methods of that? | 10:17 |
| msuszko | _uploadPack (zuul/driver/gerrit/gerritconnection.py:1302) | 10:29 |
| tobiash | msuszko: looking at the source of requessts chardet seems to be only used in the text property (and only if the encoding is not known a priori) | 10:29 |
| msuszko | my corporate proxy filters out pastebin and alike | 10:30 |
| tobiash | _uploadPack uses text as well | 10:31 |
| *** ykarel has joined #zuul | 10:31 | |
| tobiash | I guess that misses the headers | 10:31 |
| msuszko | does git-upload-pack encode its data? | 10:32 |
| *** zenkuro has quit IRC | 10:33 | |
| tobiash | it's a text format: https://git-scm.com/docs/http-protocol | 10:33 |
| *** zenkuro has joined #zuul | 10:33 | |
| msuszko | should it be decoded as ansi then? | 10:34 |
| msuszko | ascii | 10:35 |
| tobiash | this contains some info about the encoding: https://git-scm.com/docs/http-protocol#_dumb_clients | 10:35 |
| tobiash | interestingly: Clients MUST NOT attempt to validate the returned Content-Type. | 10:35 |
| msuszko | does Response.content attempt to use chardet? | 10:38 |
| tobiash | msuszko: no, I guess that's the byte array that gets decoded by Response.text | 10:40 |
| msuszko | Response.content uses .iter_content which pases decode_conten=True to .raw.stream (urllib3?): https://github.com/psf/requests/blob/master/requests/models.py#L753 | 10:48 |
| tobiash | at least content is cached within the response object | 10:52 |
| msuszko | no, You are right. Reponse.text is the only chardet use | 10:53 |
| *** wuchunyang has joined #zuul | 10:53 | |
| *** snapiri has quit IRC | 10:56 | |
| *** snapiri has joined #zuul | 11:01 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Call r.text only once https://review.opendev.org/c/zuul/zuul/+/770071 | 11:04 |
| *** jcapitao is now known as jcapitao_lunch | 11:21 | |
| msuszko | apparently only place it used Response.text for upload-pack was this line: self.iolog.debug('Received: %s %s' % (r.status_code, r.text,)) | 11:29 |
| tobiash | avass: I wonder if leaking the api key can be avoided without removing all possibilities to debug upload failures here: https://review.opendev.org/c/zuul/zuul-jobs/+/768111 | 11:38 |
| *** rfolco has joined #zuul | 11:38 | |
| *** rfolco is now known as rfolco|doctor | 11:39 | |
| msuszko | while back i changed Response.text to Reponse.content in _uploadPack, because it couldn't read branch names with non-ascii unicode | 11:43 |
| msuszko | the only line left with .text was this debug log | 11:43 |
| msuszko | and it was causing encoding detection using utterly slow chardet | 11:45 |
| *** ykarel is now known as ykarel|afk | 11:50 | |
| *** ykarel|afk has quit IRC | 11:55 | |
| *** ykarel|afk has joined #zuul | 12:07 | |
| *** wuchunyang has quit IRC | 12:09 | |
| *** wuchunyang has joined #zuul | 12:11 | |
| avass | tobiash: I'd like that too but I'm not sure how | 12:15 |
| *** wuchunyang has quit IRC | 12:15 | |
| avass | tobiash: I'm not sure about this but I think ansible also saved the headers when showing the invocation as well so that would have leaked it too. | 12:17 |
| openstackgerrit | Merged zuul/zuul-jobs master: Better error output for update-test-platforms.py https://review.opendev.org/c/zuul/zuul-jobs/+/766754 | 12:17 |
| *** ykarel|afk has quit IRC | 12:22 | |
| *** jpena is now known as jpena|lunch | 12:30 | |
| *** rlandy has joined #zuul | 12:31 | |
| *** msuszko has quit IRC | 12:39 | |
| *** msuszko has joined #zuul | 12:41 | |
| *** sduthil has joined #zuul | 12:46 | |
| *** holser has joined #zuul | 12:57 | |
| *** jcapitao_lunch is now known as jcapitao | 13:01 | |
| *** ykarel has joined #zuul | 13:06 | |
| *** ykarel_ has joined #zuul | 13:21 | |
| *** ykarel has quit IRC | 13:23 | |
| *** ykarel_ is now known as ykarel | 13:25 | |
| *** jpena|lunch is now known as jpena | 13:29 | |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Bump openshift dep https://review.opendev.org/c/zuul/nodepool/+/765873 | 13:32 |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Bump openshift dep https://review.opendev.org/c/zuul/nodepool/+/765873 | 13:44 |
| *** yoctozepto has joined #zuul | 13:45 | |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Rework quick-start and prepare for other tutorials https://review.opendev.org/c/zuul/zuul/+/732066 | 14:16 |
| *** rfolco|doctor is now known as rfolco | 14:18 | |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Bump openshift dep https://review.opendev.org/c/zuul/nodepool/+/765873 | 14:18 |
| *** jfoufas1 has quit IRC | 14:27 | |
| *** holser has quit IRC | 14:27 | |
| *** holser has joined #zuul | 14:28 | |
| *** jamesmcarthur has joined #zuul | 14:37 | |
| *** tosky has joined #zuul | 14:39 | |
| *** jfoufas1 has joined #zuul | 14:44 | |
| *** spotz has joined #zuul | 14:57 | |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Bump openshift dep https://review.opendev.org/c/zuul/nodepool/+/765873 | 15:00 |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Bump openshift dep https://review.opendev.org/c/zuul/nodepool/+/765873 | 15:24 |
| *** ykarel has quit IRC | 15:44 | |
| tobiash | clarkb: this finally passes the k8s and openshift functional tests ^ | 15:55 |
| tobiash | oh and happy new year everyone :) | 15:55 |
| avass | happy new year! :) | 16:04 |
| corvus | tobiash: happy new year! welcome back :) | 16:05 |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Bump openshift dep https://review.opendev.org/c/zuul/nodepool/+/765873 | 16:09 |
| *** hashar has quit IRC | 16:09 | |
| *** cloudnull has quit IRC | 16:11 | |
| *** cloudnull has joined #zuul | 16:14 | |
| *** zenkuro has quit IRC | 16:17 | |
| *** zenkuro has joined #zuul | 16:17 | |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Bump openshift dep https://review.opendev.org/c/zuul/nodepool/+/765873 | 16:26 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Rework quick-start and prepare for other tutorials https://review.opendev.org/c/zuul/zuul/+/732066 | 16:32 |
| *** jfoufas1 has quit IRC | 16:50 | |
| *** jpena is now known as jpena|off | 16:50 | |
| *** sduthil has quit IRC | 16:52 | |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: zuul_stream: close streamer when task is skipped https://review.opendev.org/c/zuul/zuul/+/735476 | 16:57 |
| *** rfolco has quit IRC | 16:58 | |
| *** rpittau is now known as rpittau|afk | 17:10 | |
| *** jcapitao has quit IRC | 17:14 | |
| *** rfolco has joined #zuul | 17:22 | |
| *** holser has quit IRC | 17:22 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: DNM: Test new openshift version from nodepool https://review.opendev.org/c/zuul/zuul-operator/+/770158 | 17:23 |
| *** tosky has quit IRC | 17:24 | |
| *** tosky has joined #zuul | 17:25 | |
| *** piotrowskim has quit IRC | 17:28 | |
| openstackgerrit | Jonas Sticha proposed zuul/nodepool master: aws: add support for uploading diskimages https://review.opendev.org/c/zuul/nodepool/+/735217 | 17:31 |
| *** holser has joined #zuul | 17:36 | |
| *** openstackgerrit has quit IRC | 17:37 | |
| *** evrardjp has quit IRC | 17:37 | |
| *** evrardjp has joined #zuul | 17:37 | |
| *** jamesmcarthur has quit IRC | 17:54 | |
| *** jamesmcarthur has joined #zuul | 17:55 | |
| *** jamesmcarthur has quit IRC | 18:01 | |
| *** jamesmcarthur has joined #zuul | 18:07 | |
| *** jamesmcarthur has quit IRC | 18:07 | |
| *** jamesmcarthur has joined #zuul | 18:08 | |
| *** mgoddard has quit IRC | 18:14 | |
| *** mgoddard has joined #zuul | 18:15 | |
| *** jamesmcarthur has quit IRC | 18:16 | |
| *** hamalq has joined #zuul | 18:16 | |
| *** mgoddard has quit IRC | 18:24 | |
| *** jamesmcarthur has joined #zuul | 18:29 | |
| *** sduthil has joined #zuul | 18:31 | |
| *** rfolco has quit IRC | 18:43 | |
| *** jamesmcarthur_ has joined #zuul | 18:46 | |
| *** jamesmcarthur has quit IRC | 18:50 | |
| *** rfolco has joined #zuul | 19:24 | |
| *** gmann is now known as gmann_afk | 19:25 | |
| *** saneax has quit IRC | 19:47 | |
| *** openstackgerrit has joined #zuul | 19:54 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Separate out executor server from runner https://review.opendev.org/c/zuul/zuul/+/607079 | 19:54 |
| *** zenkuro has quit IRC | 20:51 | |
| *** rlandy is now known as rlandy|biab | 20:52 | |
| *** zenkuro has joined #zuul | 20:52 | |
| *** rfolco has quit IRC | 21:04 | |
| *** gmann_afk is now known as gmann | 21:13 | |
| *** zenkuro has quit IRC | 21:19 | |
| *** zenkuro has joined #zuul | 21:19 | |
| *** jamesmcarthur_ has quit IRC | 21:34 | |
| *** jamesmcarthur has joined #zuul | 21:35 | |
| *** jamesmcarthur has quit IRC | 21:36 | |
| *** jamesmcarthur has joined #zuul | 21:36 | |
| *** jamesmcarthur has quit IRC | 21:47 | |
| *** jamesmcarthur has joined #zuul | 21:47 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Separate out executor server from runner https://review.opendev.org/c/zuul/zuul/+/607079 | 21:51 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Move repository preparation into common class https://review.opendev.org/c/zuul/zuul/+/648642 | 21:51 |
| *** jamesmcarthur has quit IRC | 22:01 | |
| *** jamesmcarthur has joined #zuul | 22:02 | |
| *** zenkuro has quit IRC | 22:05 | |
| *** zenkuro has joined #zuul | 22:06 | |
| *** rlandy|biab is now known as rlandy | 22:11 | |
| *** jamesmcarthur has quit IRC | 22:25 | |
| *** jamesmcarthur has joined #zuul | 22:26 | |
| *** jamesmcarthur has quit IRC | 22:30 | |
| *** jamesmcarthur has joined #zuul | 22:30 | |
| *** zenkuro has quit IRC | 22:49 | |
| *** zenkuro has joined #zuul | 22:50 | |
| *** jamesmcarthur has quit IRC | 22:52 | |
| *** jamesmcarthur has joined #zuul | 22:52 | |
| *** jamesmcarthur has quit IRC | 23:02 | |
| *** ikhan has quit IRC | 23:04 | |
| *** zenkuro has quit IRC | 23:06 | |
| *** zenkuro has joined #zuul | 23:06 | |
| *** icey has quit IRC | 23:08 | |
| *** icey has joined #zuul | 23:09 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Separate out executor server from runner https://review.opendev.org/c/zuul/zuul/+/607079 | 23:22 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Move repository preparation into common class https://review.opendev.org/c/zuul/zuul/+/648642 | 23:22 |
| *** jamesmcarthur has joined #zuul | 23:34 | |
| *** jamesmcarthur has quit IRC | 23:34 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!