@jim:acmegating.com | zuul-maint: how does this look for a zuul release? commit 4d6db3602fee4383bf92445e434047f77adfaa4e (HEAD -> master, tag: 10.1.0, origin/master, gerrit/master) | 00:23 |
---|---|---|
@anoop-jose:matrix.org | Is it possible to have a stand-alone zuul installed, and set Gerrit, GitHub and GitLab connected to it? I meant, if someone use Gerrit for code push and review the events must be streamed to zuul. And same for GitHub and GitLab. | 03:26 |
All these should be connected at the same time to same zuul installation. | ||
-@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 916709: Fix Gerrit query for topics with special chars https://review.opendev.org/c/zuul/zuul/+/916709 | 06:50 | |
@anoop-jose:matrix.org | * I have a stand-alone Zuul installed, and its currently connected to Gerrit. Is it also possible to connect GitHub and GitLab connect to the same Zuul installation? | 06:50 |
I meant, if someone uses Gerrit for code push and review the events must be streamed to Zuul. And the same for GitHub and GitLab. All these should be connected at the same time to the same Zuul installation. | ||
-@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 916709: Fix Gerrit query for topics with special chars https://review.opendev.org/c/zuul/zuul/+/916709 | 06:55 | |
@sjal:matrix.org | > <@anoop-jose:matrix.org> I have a stand-alone Zuul installed, and its currently connected to Gerrit. Is it also possible to connect GitHub and GitLab connect to the same Zuul installation? | 07:26 |
> I meant, if someone uses Gerrit for code push and review the events must be streamed to Zuul. And the same for GitHub and GitLab. All these should be connected at the same time to the same Zuul installation. | ||
you can setup different connections in the config, it won't stop you from adding several different ones. Each project is tied to a different connection also | ||
@sjal:matrix.org | gerrit/gitlab etc. are just drivers for those connections | 07:28 |
-@gerrit:opendev.org- Felix Edel proposed: | 08:22 | |
- [zuul/zuul] 916284: Implement new status page https://review.opendev.org/c/zuul/zuul/+/916284 | ||
- [zuul/zuul] 916285: Make helper functions available to other components https://review.opendev.org/c/zuul/zuul/+/916285 | ||
- [zuul/zuul] 916286: Implement QueueItemPopover https://review.opendev.org/c/zuul/zuul/+/916286 | ||
- [zuul/zuul] 916287: Implement PipelineDetails view https://review.opendev.org/c/zuul/zuul/+/916287 | ||
- [zuul/zuul] 916288: Sort pipelines on status page by number of queue items https://review.opendev.org/c/zuul/zuul/+/916288 | ||
- [zuul/zuul] 916289: Align QueueItem and QueueItemPopover https://review.opendev.org/c/zuul/zuul/+/916289 | ||
- [zuul/zuul] 916290: Align job progress bars and job result labels https://review.opendev.org/c/zuul/zuul/+/916290 | ||
- [zuul/zuul] 916744: Visualize branches in ChangeQueues https://review.opendev.org/c/zuul/zuul/+/916744 | ||
@drichardt:matrix.org | Hi guys. After moving our Zuul to new domain and updateing certs and everything I'm getting strange error message from zuul-web | 08:42 |
``` | ||
2024-04-23 08:21:28,893 INFO zuul.GithubConnection.GithubClientManager: Authing to GitHub | ||
Traceback (most recent call last): | ||
File "/usr/local/lib/python3.10/site-packages/jwt/algorithms.py", line 257, in prepare_key | ||
key = load_pem_private_key(key, password=None) | ||
File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/primitives/serialization/base.py", line 24, in load_pem_private_key | ||
return ossl.load_pem_private_key( | ||
File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 949, in load_pem_private_key | ||
return self._load_key( | ||
File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1201, in _load_key | ||
self._handle_key_loading_error() | ||
File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1257, in _handle_key_loading_error | ||
raise ValueError( | ||
ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=503841036, lib=60, reason=524556, reason_text=b'error:1E08010C:DECODER routines::unsupported')]) | ||
During handling of the above exception, another exception occurred: | ||
Traceback (most recent call last): | ||
File "/usr/local/bin/zuul-scheduler", line 8, in <module> | ||
sys.exit(main()) | ||
File "/usr/local/lib/python3.10/site-packages/zuul/cmd/scheduler.py", line 127, in main | ||
Scheduler().main() | ||
File "/usr/local/lib/python3.10/site-packages/zuul/cmd/__init__.py", line 248, in main | ||
self.run() | ||
File "/usr/local/lib/python3.10/site-packages/zuul/cmd/scheduler.py", line 92, in run | ||
self.connections.load(self.sched.zk_client, | ||
File "/usr/local/lib/python3.10/site-packages/zuul/lib/connections.py", line 82, in load | ||
connection.onLoad(zk_client, component_registry) | ||
File "/usr/local/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 1344, in onLoad | ||
self._github_client_manager.initialize() | ||
File "/usr/local/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 993, in initialize | ||
self._prime_installation_map() | ||
File "/usr/local/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 1159, in _prime_installation_map | ||
headers = self._get_app_auth_headers() | ||
File "/usr/local/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 1081, in _get_app_auth_headers | ||
app_token = jwt.encode(data, | ||
File "/usr/local/lib/python3.10/site-packages/jwt/api_jwt.py", line 67, in encode | ||
return api_jws.encode(json_payload, key, algorithm, headers, json_encoder) | ||
File "/usr/local/lib/python3.10/site-packages/jwt/api_jws.py", line 153, in encode | ||
key = alg_obj.prepare_key(key) | ||
File "/usr/local/lib/python3.10/site-packages/jwt/algorithms.py", line 259, in prepare_key | ||
key = load_pem_public_key(key) | ||
File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/primitives/serialization/base.py", line 34, in load_pem_public_key | ||
return ossl.load_pem_public_key(data) | ||
File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 996, in load_pem_public_key | ||
self._handle_key_loading_error() | ||
File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1257, in _handle_key_loading_error | ||
raise ValueError( | ||
ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=75497580, lib=9, reason=108, reason_text=b'error:0480006C:PEM routines::no start line')]) | ||
``` | ||
does someone of you has an Idea which key he tries to use there and what could cause this error ? | ||
-@gerrit:opendev.org- Felix Edel proposed: [zuul/zuul] 916744: Visualize branches in ChangeQueues https://review.opendev.org/c/zuul/zuul/+/916744 | 08:44 | |
-@gerrit:opendev.org- Felix Edel proposed: | 09:17 | |
- [zuul/zuul] 916287: Implement PipelineDetails view https://review.opendev.org/c/zuul/zuul/+/916287 | ||
- [zuul/zuul] 916288: Sort pipelines on status page by number of queue items https://review.opendev.org/c/zuul/zuul/+/916288 | ||
- [zuul/zuul] 916289: Align QueueItem and QueueItemPopover https://review.opendev.org/c/zuul/zuul/+/916289 | ||
- [zuul/zuul] 916290: Align job progress bars and job result labels https://review.opendev.org/c/zuul/zuul/+/916290 | ||
- [zuul/zuul] 916744: Visualize branches in ChangeQueues https://review.opendev.org/c/zuul/zuul/+/916744 | ||
@drichardt:matrix.org | > <@drichardt:matrix.org> Hi guys. After moving our Zuul to new domain and updateing certs and everything I'm getting strange error message from zuul-web | 09:33 |
> | ||
> ``` | ||
> 2024-04-23 08:21:28,893 INFO zuul.GithubConnection.GithubClientManager: Authing to GitHub | ||
> Traceback (most recent call last): | ||
> File "/usr/local/lib/python3.10/site-packages/jwt/algorithms.py", line 257, in prepare_key | ||
> key = load_pem_private_key(key, password=None) | ||
> File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/primitives/serialization/base.py", line 24, in load_pem_private_key | ||
> return ossl.load_pem_private_key( | ||
> File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 949, in load_pem_private_key | ||
> return self._load_key( | ||
> File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1201, in _load_key | ||
> self._handle_key_loading_error() | ||
> File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1257, in _handle_key_loading_error | ||
> raise ValueError( | ||
> ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=503841036, lib=60, reason=524556, reason_text=b'error:1E08010C:DECODER routines::unsupported')]) | ||
> During handling of the above exception, another exception occurred: | ||
> Traceback (most recent call last): | ||
> File "/usr/local/bin/zuul-scheduler", line 8, in <module> | ||
> sys.exit(main()) | ||
> File "/usr/local/lib/python3.10/site-packages/zuul/cmd/scheduler.py", line 127, in main | ||
> Scheduler().main() | ||
> File "/usr/local/lib/python3.10/site-packages/zuul/cmd/__init__.py", line 248, in main | ||
> self.run() | ||
> File "/usr/local/lib/python3.10/site-packages/zuul/cmd/scheduler.py", line 92, in run | ||
> self.connections.load(self.sched.zk_client, | ||
> File "/usr/local/lib/python3.10/site-packages/zuul/lib/connections.py", line 82, in load | ||
> connection.onLoad(zk_client, component_registry) | ||
> File "/usr/local/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 1344, in onLoad | ||
> self._github_client_manager.initialize() | ||
> File "/usr/local/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 993, in initialize | ||
> self._prime_installation_map() | ||
> File "/usr/local/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 1159, in _prime_installation_map | ||
> headers = self._get_app_auth_headers() | ||
> File "/usr/local/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 1081, in _get_app_auth_headers | ||
> app_token = jwt.encode(data, | ||
> File "/usr/local/lib/python3.10/site-packages/jwt/api_jwt.py", line 67, in encode | ||
> return api_jws.encode(json_payload, key, algorithm, headers, json_encoder) | ||
> File "/usr/local/lib/python3.10/site-packages/jwt/api_jws.py", line 153, in encode | ||
> key = alg_obj.prepare_key(key) | ||
> File "/usr/local/lib/python3.10/site-packages/jwt/algorithms.py", line 259, in prepare_key | ||
> key = load_pem_public_key(key) | ||
> File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/primitives/serialization/base.py", line 34, in load_pem_public_key | ||
> return ossl.load_pem_public_key(data) | ||
> File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 996, in load_pem_public_key | ||
> self._handle_key_loading_error() | ||
> File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1257, in _handle_key_loading_error | ||
> raise ValueError( | ||
> ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=75497580, lib=9, reason=108, reason_text=b'error:0480006C:PEM routines::no start line')]) | ||
> ``` | ||
> | ||
> does someone of you has an Idea which key he tries to use there and what could cause this error ? | ||
Found issue. Certifikate was not in pem format. Creating new app credential and adding it solved issue | ||
@f2ked:matrix.org | how is [`zuul.executor.work_root`](https://zuul-ci.org/docs/zuul/latest/job-content.html#var-zuul.executor.work_root) specified? Docs seem to suggest it should be `${HOME}/work` but my executors are using `$HOME`. | 14:02 |
@jim:acmegating.com | f2ked: can you link to the docs that suggest it should be $home/work? | 14:03 |
-@gerrit:opendev.org- Christian Mueller proposed: [zuul/nodepool] 916801: WIP: enable EC2 Fleet API https://review.opendev.org/c/zuul/nodepool/+/916801 | 15:12 | |
-@gerrit:opendev.org- Christian Mueller proposed: [zuul/nodepool] 916801: WIP: enable EC2 Fleet API https://review.opendev.org/c/zuul/nodepool/+/916801 | 15:28 | |
@f2ked:matrix.org | top of the same page | 15:31 |
https://zuul-ci.org/docs/zuul/latest/job-content.html#working-directory | ||
@f2ked:matrix.org | it doesn't talk about `$HOME` but I'm assuming it is wherever ssh | 15:33 |
@f2ked:matrix.org | * it doesn't talk about `$HOME` but I'm assuming it is wherever ssh'in lands you | 15:33 |
@jim:acmegating.com | ssh isn't involved here; the executor is the host that ssh is executed *from*, not *to*. | 15:38 |
thanks for the pointer to that text. i could see why one might assume that was in relation to $HOME, but the text actually says `work/` is under "a directory to hold all the content related to the job" -- internally, that's the build directory. we set $HOME to the build directory + work/ because it's the highest level directory writable by the job. we could explicitly add a note about $HOME, however, i would advise against using $HOME and just consider the behavior undefined, and instead just use the zuul variables for generating paths. | ||
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 915091: Add --disable-pipelines option https://review.opendev.org/c/zuul/zuul/+/915091 | 16:42 | |
@f2ked:matrix.org | ok.. but I'm still confused (could be terminology) | 18:43 |
I have a worker node that I get from a static node pool. | ||
when the source "appears" on that node it is in `./src` not `./work/src` as this document suggests | ||
Associating `.` with `$HOME` is me. | ||
I guess my real question is, how can I specify the directory where all "work" will be done on that node? | ||
@jim:acmegating.com | f2ked: yeah i think there's a terminology confusion. `executor` means the zuul-executor process. that's where the ansible-playbook process is run, and the ssh connection to the remote worker node originates from. that's also where the zuul.executor.* ansible variables in the docs are relevant. | 18:58 |
the worker node is the static node that receives the ssh connection. strictly speaking, the responsibility of the zuul program ends there, and what happens next is up to the user, but we generally expect users to use the roles in the zuul-jobs repo, and by shared convention, they copy the git source code from the executor to the remote node and put it it `$HOME/src/...`. essentially they mirror `.../work/src/` on the executor to `$HOME/src/` on the remote worker node. | ||
@picog:matrix.org | I'm trying to debug a very weird job where there seems to be no output of the job and the build seems to fail for no reason. | 19:20 |
Any clues what I should be looking for? | ||
If I grep for the event-id of the job, I see: | ||
``` | ||
a718dabae24b 2024-04-23 16:54:02,499 INFO zuul.AnsibleJob: [e: 374b69fc2bd14c6f8d1582013faad159] [build: 2bd641dd880d49adab1d23309d09645b] Checking out gerrit/zuul/zuul-jobs branch master | ||
a718dabae24b ---------------------------------------- | ||
a718dabae24b Exception occurred during processing of request from ('::ffff:10.89.0.149', 48860, 0, 0) | ||
a718dabae24b Traceback (most recent call last): | ||
a718dabae24b File "/usr/local/lib/python3.11/site-packages/zuul/lib/log_streamer.py", line 77, in handle | ||
a718dabae24b self.stream_log(log_file) | ||
a718dabae24b File "/usr/local/lib/python3.11/site-packages/zuul/lib/log_streamer.py", line 98, in stream_log | ||
a718dabae24b if self.follow_log(log): | ||
a718dabae24b ^^^^^^^^^^^^^^^^^^^^ | ||
a718dabae24b File "/usr/local/lib/python3.11/site-packages/zuul/lib/log_streamer.py", line 144, in follow_log | ||
a718dabae24b ret = self.request.recv(1024) | ||
a718dabae24b ^^^^^^^^^^^^^^^^^^^^^^^ | ||
a718dabae24b ConnectionResetError: [Errno 104] Connection reset by peer | ||
a718dabae24b | ||
a718dabae24b During handling of the above exception, another exception occurred: | ||
a718dabae24b | ||
a718dabae24b Traceback (most recent call last): | ||
a718dabae24b File "/usr/local/lib/python3.11/socketserver.py", line 691, in process_request_thread | ||
a718dabae24b self.finish_request(request, client_address) | ||
a718dabae24b File "/usr/local/lib/python3.11/socketserver.py", line 361, in finish_request | ||
a718dabae24b self.RequestHandlerClass(request, client_address, self) | ||
a718dabae24b File "/usr/local/lib/python3.11/socketserver.py", line 755, in __init__ | ||
a718dabae24b self.handle() | ||
a718dabae24b File "/usr/local/lib/python3.11/site-packages/zuul/lib/log_streamer.py", line 82, in handle | ||
a718dabae24b self.request.sendall(msg.encode("utf-8")) | ||
a718dabae24b BrokenPipeError: [Errno 32] Broken pipe | ||
a718dabae24b ---------------------------------------- | ||
a718dabae24b 2024-04-23 17:14:39,876 INFO zuul.AnsibleJob: [e: 374b69fc2bd14c6f8d1582013faad159] [build: cab6dce863a44eaeb2083b4f1477b8e1] Early failure in job | ||
a718dabae24b 2024-04-23 17:14:48,329 INFO zuul.ExecutorServer: [e: 374b69fc2bd14c6f8d1582013faad159] [build: cab6dce863a44eaeb2083b4f1477b8e1] Held status set to False | ||
a718dabae24b 2024-04-23 17:14:50,327 INFO zuul.nodepool: [e: 374b69fc2bd14c6f8d1582013faad159] Returning nodeset <NodeSet [<Node 0000000891 ['sslva-builder']:node>]> | ||
a718dabae24b 2024-04-23 17:14:50,449 INFO zuul.nodepool: [e: 374b69fc2bd14c6f8d1582013faad159] Nodeset <NodeSet [<Node 0000000891 ['sslva-builder']:node>]> with 1 nodes was in use for 1253.2204751209356 seconds for build <BuildRequest cab6dce863a44eaeb2083b4f1477b8e1, job=293e9d082a83470c930c74fa66bd6be1, state=completed, path=/zuul/executor/unzoned/requests/cab6dce863a44eaeb2083b4f1477b8e1 zone=None> for project gerrit.netronome.com/nse/sslva-build | ||
a718dabae24b 2024-04-23 17:14:50,449 INFO zuul.AnsibleJob: [e: 374b69fc2bd14c6f8d1582013faad159] [build: cab6dce863a44eaeb2083b4f1477b8e1] Job execution took: 1253.220 seconds | ||
a718dabae24b 2024-04-23 17:23:25,910 INFO zuul.ExecutorServer: [e: 374b69fc2bd14c6f8d1582013faad159] [build: 2bd641dd880d49adab1d23309d09645b] Held status set to False | ||
a718dabae24b 2024-04-23 17:23:26,649 INFO zuul.nodepool: [e: 374b69fc2bd14c6f8d1582013faad159] Returning nodeset <NodeSet [<Node 0000000894 ['sslva-builder']:oracle9>]> | ||
a718dabae24b 2024-04-23 17:23:26,686 INFO zuul.nodepool: [e: 374b69fc2bd14c6f8d1582013faad159] Nodeset <NodeSet [<Node 0000000894 ['sslva-builder']:oracle9>]> with 1 nodes was in use for 1771.3888368420303 seconds for build <BuildRequest 2bd641dd880d49adab1d23309d09645b, job=4f43e36fd87047bba203dc4aa4baa2fe, state=completed, path=/zuul/executor/unzoned/requests/2bd641dd880d49adab1d23309d09645b zone=None> for project gerrit.netronome.com/nse/sslva-build | ||
a718dabae24b 2024-04-23 17:23:26,686 INFO zuul.AnsibleJob: [e: 374b69fc2bd14c6f8d1582013faad159] [build: 2bd641dd880d49adab1d23309d09645b] Job execution took: 1771.389 seconds | ||
``` | ||
@picog:matrix.org | Is this log streamer error something that I should be worried about? | 19:21 |
@sylvass:albinvass.se | I think that is the zuul-executor trying to connect to the log-streaming on the remote node, but eventually fails. I don't think that should be something critical, but running this role early in the job should give you live log streaming for shell and command tasks: | 19:27 |
https://opendev.org/opendev/base-jobs/src/branch/master/playbooks/base/pre.yaml#L45 | ||
@picog:matrix.org | > <@sylvass:albinvass.se> I think that is the zuul-executor trying to connect to the log-streaming on the remote node, but eventually fails. I don't think that should be something critical, but running this role early in the job should give you live log streaming for shell and command tasks: | 19:32 |
> https://opendev.org/opendev/base-jobs/src/branch/master/playbooks/base/pre.yaml#L45 | ||
Thanks, will add that. | ||
-@gerrit:opendev.org- Zuul merged on behalf of Dr. Jens Harbott: [zuul/nodepool] 916053: Make debootstrap support noble https://review.opendev.org/c/zuul/nodepool/+/916053 | 20:27 | |
@f2ked:matrix.org | when running with the tutorial setup, should I expect | 21:06 |
``` | ||
http://XXX:9000/t/example-tenant/stream/XXX?logfile=console.log | ||
``` | ||
to show more than | ||
``` | ||
--- END OF STREAM --- | ||
``` | ||
while the job is running? | ||
@clarkb:matrix.org | Your base job needs to run the role that starts the log streamer daemon | 21:16 |
@clarkb:matrix.org | Like https://opendev.org/opendev/base-jobs/src/branch/master/playbooks/base/pre.yaml#L45 | 21:18 |
-@gerrit:opendev.org- dasm proposed: [zuul/zuul-client] 916845: [zuulclient] Implemented buildsets api https://review.opendev.org/c/zuul/zuul-client/+/916845 | 21:57 | |
@f2ked:matrix.org | could you point me to the clause that does it? | 22:17 |
the [docs](https://zuul-ci.org/docs/zuul/latest/developer/ansible.html#capturing-live-command-output) say | ||
> The global job configuration should run the zuul_console task as a very early prerequisite step. | ||
so is it a task or a role? | ||
@f2ked:matrix.org | I guess it's | 22:21 |
``` | ||
- hosts: all | ||
roles: | ||
- add-build-sshkey | ||
+ - start-zuul-console | ||
- prepare-workspace | ||
``` | ||
@clarkb:matrix.org | The line I linked to is the role `start-zuul-console` | 22:21 |
@clarkb:matrix.org | There is an underlying task that does it within the role whcih is what you actually need to run. But the role encapsulates it nicely for you | 22:22 |
@f2ked:matrix.org | oops, I see that highlight now 😬 | 22:22 |
@f2ked:matrix.org | Clark: that did seem to be enough :( still only see | 22:38 |
``` | ||
--- END OF STREAM --- | ||
``` | ||
@f2ked:matrix.org | and yes port 19885 on the worker node is accessible | 22:40 |
@clarkb:matrix.org | Maybe look at the zuul executor debug log to see if the ansible output there indicates why it may be failing | 22:43 |
@clarkb:matrix.org | also ensure the hosts you run that on are `all` not `localhost` | 22:43 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!