| *** ianychoi has joined #airshipit | 00:19 | |
| *** cheng1 has quit IRC | 01:53 | |
| *** cheng1 has joined #airshipit | 01:54 | |
| openstackgerrit | Kaspars Skels proposed airship/treasuremap master: Seaworthy pipeline fixes https://review.opendev.org/665272 | 01:56 |
|---|---|---|
| openstackgerrit | Kaspars Skels proposed airship/treasuremap master: Seaworthy pipeline fixes https://review.opendev.org/665272 | 01:57 |
| *** michael-beaver has quit IRC | 02:43 | |
| *** nishantkr has quit IRC | 03:07 | |
| *** adrianreza_ has quit IRC | 03:07 | |
| *** mugsie has quit IRC | 03:22 | |
| *** cheng1 has quit IRC | 03:22 | |
| *** AlexNoskov has quit IRC | 03:22 | |
| *** kskels has quit IRC | 03:22 | |
| *** jezogwza has quit IRC | 03:22 | |
| *** rustyweise has quit IRC | 03:22 | |
| *** dasp has quit IRC | 03:22 | |
| *** licanwei has quit IRC | 03:32 | |
| *** mugsie has joined #airshipit | 03:37 | |
| *** cheng1 has joined #airshipit | 03:38 | |
| *** AlexNoskov has joined #airshipit | 03:38 | |
| *** kskels has joined #airshipit | 03:38 | |
| *** jezogwza has joined #airshipit | 03:38 | |
| *** rustyweise has joined #airshipit | 03:38 | |
| *** dasp has joined #airshipit | 03:38 | |
| *** irclogbot_3 has quit IRC | 03:41 | |
| *** irclogbot_2 has joined #airshipit | 03:41 | |
| *** dasp has quit IRC | 03:58 | |
| *** AlexNoskov has quit IRC | 04:21 | |
| *** dasp has joined #airshipit | 04:21 | |
| *** sgrasley has quit IRC | 04:50 | |
| *** kskels has quit IRC | 05:43 | |
| *** pgaxatte has joined #airshipit | 06:27 | |
| *** mikkos has joined #airshipit | 06:48 | |
| openstackgerrit | Merged airship/treasuremap master: [aiab] Fix readme formatting https://review.opendev.org/665019 | 07:30 |
| *** mbologna has quit IRC | 08:22 | |
| *** mbologna has joined #airshipit | 08:25 | |
| *** pkaralis has joined #airshipit | 08:47 | |
| *** roman_g has quit IRC | 08:55 | |
| *** amalrajgenocidex has joined #airshipit | 10:10 | |
| openstackgerrit | Kudaka Poorna Rajesh proposed airship/promenade master: Add liveness/readiness probes to calico-etcd https://review.opendev.org/659863 | 10:10 |
| amalrajgenocidex | Hi, I'm getting below error on Airsloop airship. Any idea | 10:10 |
| amalrajgenocidex | root@airsloop-control-1:~/treasuremap# tools/airship shipyard create configdocs design --directory=/target/collect Error: Unable to invoke action due to: The account is locked for user: 47050a307ad74d9087d6c095f930fa20. (HTTP 401) (Request-ID: req-d29bcbd3-3c54-4c6a-802c-8ade5ee2863b) | 10:10 |
| amalrajgenocidex | Leave, found the issue | 10:13 |
| openstackgerrit | Kudaka Poorna Rajesh proposed airship/promenade master: Add liveness/readiness probes to calico-etcd https://review.opendev.org/659863 | 10:37 |
| *** amalrajgenocidex has quit IRC | 11:12 | |
| rustyweise | Hey guys, I'm deploying the airsloop environment and am hitting a failure when running genesis.sh airship-kubernetes-calico-etcd-etcd-test container and when dumping the logs for that pod, its pretty-terse response is: | 11:59 |
| rustyweise | ubuntu@airsloop-control-1:~/airshipit/treasuremap$ kubectl logs airship-kubernetes-calico-etcd-etcd-test --namespace kube-system | 12:00 |
| rustyweise | https://10.96.232.136:6666 is unhealthy: failed to connect: context deadline exceeded | 12:00 |
| rustyweise | Error: unhealthy cluster | 12:00 |
| rustyweise | ubuntu@airsloop-control-1:~/airshipit/treasuremap$ 2019-06-14 11:56:17.831 409 INFO armada.handlers.lock [-] Acquiring lock | 12:00 |
| rustyweise | 2019-06-14 11:56:18.127 409 INFO armada.handlers.armada [-] Performing pre-flight operations. | 12:00 |
| *** amalrajgenocidex has joined #airshipit | 12:04 | |
| amalrajgenocidex | Is there any option to delete depolyed sites. | 12:07 |
| amalrajgenocidex | I tried deployment multiple times and it failed. But it still shows in "tools/shipyard get actions" | 12:07 |
| rustyweise | Here's a anonymized pastebin (kinda pointless by now *shrug*) of the diff of my airshipit/treasuremap repo https://pastebin.com/UjcMQLkL | 12:16 |
| openstackgerrit | Merged airship/treasuremap master: Add Genesis network example to Airsloop docs https://review.opendev.org/665000 | 12:20 |
| *** amalrajgenocidex has quit IRC | 12:32 | |
| *** amalrajgenocidex has joined #airshipit | 12:42 | |
| amalrajgenocidex | Is there any option to delete depolyed sites. | 12:43 |
| amalrajgenocidex | I tried deployment multiple times and it failed. But it still shows in "tools/shipyard get actions" | 12:43 |
| amalrajgenocidex | I tried deployment multiple times and it failed. But it still shows in "tools/shipyard get actions" | 12:43 |
| openstackgerrit | Merged airship/treasuremap master: Revert ingress/ceph-rgw before emptydirs fix https://review.opendev.org/665206 | 12:47 |
| *** howell has joined #airshipit | 12:57 | |
| *** aaronsheffield has joined #airshipit | 12:58 | |
| rustyweise | I had to delete that pastebin...*sigh* alas, I continue to hit failures with deploying the test container due to what appears to be a failure to get a response from the etcd endpoint | 13:07 |
| rustyweise | it _might_ be a proxy problem, but I have proxies and no_proxies set in what I _think_ are all the right places... | 13:08 |
| rustyweise | assuming that CIDR notation works in no_proxy strings | 13:08 |
| *** kskels has joined #airshipit | 13:21 | |
| amalrajgenocidex | Hi kskels, Is there any option to delete depolyed sites. | 13:43 |
| *** sthussey has joined #airshipit | 13:47 | |
| openstackgerrit | Ian Pittwood proposed airship/spyglass master: Adds unit tests for Spyglass CLI https://review.opendev.org/663140 | 13:53 |
| *** nishantkr has joined #airshipit | 13:53 | |
| kskels | amalrajgenocidex: you mean completely re-deploy site afterwards? | 13:55 |
| kskels | or just a specific nodes, etc? | 13:55 |
| kskels | that would be re-deploy airsloop both control/compute nodes | 13:56 |
| amalrajgenocidex | I ran the deploy site command 2 times and I can see site in "tools/shipyard get actions" | 13:57 |
| amalrajgenocidex | but both installation failed. So I need to delete them and redeploy | 13:58 |
| kskels | ok - so once you have the airsloop control up you can keep re-trying installation | 13:58 |
| kskels | if you open MAAS interface | 13:59 |
| kskels | http://genesis-ip:30001/MAAS | 13:59 |
| kskels | you can check if the node did show up in MAAS | 13:59 |
| kskels | if it didn't - you don't really need to clean up anything at that point - that just means that likely bare-metal config had some error | 14:00 |
| kskels | have a look at that point to drydock logs | 14:00 |
| kskels | kubectl logs -n ucp <drydock pod> | 14:00 |
| kskels | you can also see if node part worked with | 14:01 |
| kskels | sudo kubectl get nodes -o wide | 14:01 |
| kskels | and see if the bare-metal node did deploy and come up | 14:01 |
| kskels | (airhsip instalation is structured in 2 major steps - 1 bare-metal deploy, and then software/chart deployment) | 14:02 |
| openstackgerrit | Kaspars Skels proposed airship/treasuremap master: [aiab] Allow to run aiab deployment from any directory https://review.opendev.org/664992 | 14:04 |
| amalrajgenocidex | Thanks kskels, What is the default maas dashboard credentials. I have't changed them before deployment | 14:04 |
| kskels | admin/airsloop123 | 14:05 |
| amalrajgenocidex | Thanks. Let me try again | 14:07 |
| kskels | just a re-try of deploy will likely not help - have a look at drydock log see if you find any error/issue | 14:11 |
| kskels | also under the Subnets section in MAAS - you will find networks | 14:11 |
| kskels | compare if those match to what you expect/configured | 14:11 |
| kskels | https://usercontent.irccloud-cdn.com/file/SWGtXjx1/Screenshot%20from%202019-06-14%2009-12-12.png | 14:12 |
| kskels | you should see something like this | 14:12 |
| kskels | if these networks didn't show up - this means there was some error in networking, etc | 14:13 |
| openstackgerrit | Ian Pittwood proposed airship/spyglass-plugin-xls master: Add unit tests for xls plugin CLI https://review.opendev.org/663142 | 14:14 |
| amalrajgenocidex | Ok. I will take a look | 14:16 |
| *** pgaxatte has quit IRC | 14:39 | |
| openstackgerrit | Ian Pittwood proposed airship/spyglass master: Data objects for Spyglass https://review.opendev.org/658917 | 14:47 |
| openstackgerrit | Ian Pittwood proposed airship/spyglass master: [WIP] Implements data object models https://review.opendev.org/662092 | 14:47 |
| *** lemko has joined #airshipit | 14:50 | |
| *** pkaralis has quit IRC | 14:55 | |
| rustyweise | Hi all, I am still struggling to get through genesis.sh for the first time. I believe the key challenge is the (unavoidable) proxy config. The first failure is in the 'airship-kubernetes-etcd-etcd-test' pod and the transcript/logs become a trainwreck after | 15:08 |
| kskels | there has been folks running via proxy | 15:22 |
| kskels | https://www.irccloud.com/pastebin/t9yPO48N/ | 15:22 |
| kskels | I wonder if you may need to add a few more IPs to the no_proxy | 15:23 |
| kskels | you may also check additionally tail -f /var/log/syslog | 15:23 |
| kskels | it sometimes prints something interesting | 15:23 |
| rustyweise | I've been relying on CIDR notation in the no_proxy line, in hopes of it being honored | 15:24 |
| kskels | it's not | 15:24 |
| kskels | I'm pretty sure no_proxy doesn't support that | 15:24 |
| rustyweise | *sigh* | 15:24 |
| kskels | some weird linux thing | 15:24 |
| kskels | not so much of airship doing | 15:24 |
| rustyweise | so list out all the individual addresses | 15:24 |
| kskels | usually you don't really need to include that much | 15:24 |
| rustyweise | yeah, comma, understood | 15:24 |
| kskels | it's been a while since I ran behind proxy | 15:25 |
| rustyweise | etcd ip | 15:25 |
| rustyweise | I'm assuming... | 15:25 |
| rustyweise | and the calico address on the genesis node? | 15:25 |
| rustyweise | it's highly discouraged ;) | 15:26 |
| rustyweise | 'tis not great craic | 15:26 |
| kskels | 1 sec | 15:26 |
| kskels | I'll be right back | 15:26 |
| openstackgerrit | Evgeniy L proposed airship/treasuremap master: Uplift PostgreSQL https://review.opendev.org/665228 | 15:40 |
| *** amalrajgenocidex has quit IRC | 15:55 | |
| openstackgerrit | Ian Pittwood proposed airship/spyglass master: [WIP] Implements data object models https://review.opendev.org/662092 | 15:58 |
| *** howell is now known as starboy | 16:09 | |
| *** starboy is now known as howell | 16:14 | |
| *** mikkos has quit IRC | 16:14 | |
| kskels | rustyweise: please also share the pod logs you are getting for etcd test | 16:18 |
| kskels | wonder if that could also be some other issue then just proxy - as people have used proxy for deployments not that long ago | 16:19 |
| *** howell is now known as starboy | 16:23 | |
| *** starboy is now known as howell | 16:25 | |
| rustyweise | root@airsloop-control-1:/home/ubuntu# kubectl logs airship-kubernetes-etcd-etcd-test --namespace kube-system | 16:32 |
| rustyweise | https://10.96.0.2:2379 is unhealthy: failed to connect: context deadline exceeded | 16:32 |
| rustyweise | Error: unhealthy cluster | 16:32 |
| kskels | you may be getting this also due to non-matching certs | 16:33 |
| kskels | can you login to etcd pod and do 'etcdctl member list'? | 16:33 |
| kskels | kubectl exec -it -n kube-system `etcd-pod` sh | 16:33 |
| kskels | I think there is also debug option if I remember right - `etcdctl --debug member list` or similar | 16:34 |
| kskels | also - when you generated the certs | 16:35 |
| kskels | you must delete the existing ones from the secrets | 16:35 |
| kskels | and only thing that you should change in the PKI file is the calico IP | 16:35 |
| kskels | e.g. `10.22.72.21` should be replaced to the IP that is relevant for the genesis server | 16:36 |
| kskels | and similarly for other nodes | 16:36 |
| kskels | any other IPs should be left as is - especially for etcd config | 16:36 |
| kskels | https://www.irccloud.com/pastebin/PcWAPoMI/ | 16:36 |
| kskels | change only the `10.22.72.21` IP | 16:37 |
| kskels | sort of general rule through the YAMLs for airsloop - only change the `10.22.*` networks | 16:38 |
| rustyweise | I'll try the etcd member list now | 16:43 |
| rustyweise | I can't share logs as it's against security practices *sigh* | 16:44 |
| rustyweise | here's the member list | 16:50 |
| rustyweise | 5a957a5a80f2e27, started, auxiliary-1, https://172.22.35.110:22380, https://172.22.35.110:22379 | 16:50 |
| rustyweise | 6984aa608d91cce3, started, airsloop-control-1, https://172.22.35.110:2380, https://172.22.35.110:2379 | 16:50 |
| rustyweise | e3711b00b78d9e2a, started, auxiliary-0, https://172.22.35.110:12380, https://172.22.35.110:12379 | 16:50 |
| rustyweise | OK...the calico IP is a key point as I followed the docs and added/updated the 6 entries which are read to be included, as such --> | 16:53 |
| rustyweise | # NEWSITE-CHANGEME: The following should be a list of the control plane | 16:53 |
| rustyweise | # nodes in the environment, including genesis. | 16:53 |
| rustyweise | # For each node, the `hosts` list should be comprised of: | 16:53 |
| rustyweise | # 1. The node's hostname, as already defined in baremetal/nodes.yaml | 16:53 |
| rustyweise | # 2. The node's oam IP address, as already defined in baremetal/nodes.yaml | 16:53 |
| rustyweise | # 3. The node's Calico IP address, as already defined in baremetal/nodes.yaml | 16:53 |
| rustyweise | # 4. 127.0.0.1 | 16:53 |
| rustyweise | # 5. localhost | 16:53 |
| rustyweise | # 6. kubernetes-etcd.kube-system.svc.cluster.local | 16:53 |
| *** howell has quit IRC | 16:53 | |
| rustyweise | I have to start the weekend...hopefully I can get back to this on Monday | 16:54 |
| *** howell has joined #airshipit | 16:54 | |
| rustyweise | Thanks for your help folks! slán! *wave* | 16:54 |
| *** howell has quit IRC | 16:55 | |
| *** howell has joined #airshipit | 16:56 | |
| kskels | yes - member list looks good, have a good weekend!! | 16:56 |
| kskels | port seems different | 16:56 |
| kskels | never mind - it's correct - https://172.22.35.110:2380, https://172.22.35.110:2379 | 16:57 |
| kskels | wonder if this also could be related to calico | 16:57 |
| kskels | anyway! enjoy your weekend | 16:57 |
| *** lemko has quit IRC | 17:00 | |
| *** AlexNoskov has joined #airshipit | 17:03 | |
| openstackgerrit | Merged airship/treasuremap master: Seaworthy pipeline fixes https://review.opendev.org/665272 | 17:06 |
| *** mikkos has joined #airshipit | 17:19 | |
| openstackgerrit | Merged airship/treasuremap master: [aiab] Allow to run aiab deployment from any directory https://review.opendev.org/664992 | 17:30 |
| openstackgerrit | Evgeniy L proposed airship/treasuremap master: Uplift PostgreSQL https://review.opendev.org/665228 | 17:31 |
| openstackgerrit | Evgeniy L proposed airship/treasuremap master: [aiab] Add a site linting gate to Zuul https://review.opendev.org/665007 | 17:32 |
| *** kranthikirang has joined #airshipit | 17:48 | |
| *** graysonh has joined #airshipit | 18:15 | |
| *** mikkos has quit IRC | 18:23 | |
| openstackgerrit | Ian Pittwood proposed airship/spyglass master: Data objects for Spyglass https://review.opendev.org/658917 | 18:23 |
| openstackgerrit | Ian Pittwood proposed airship/spyglass master: [WIP] Implements data object models https://review.opendev.org/662092 | 18:27 |
| openstackgerrit | Stas Egorov proposed airship/airship-in-a-bottle master: Update demo for new promenade https://review.opendev.org/665453 | 18:45 |
| openstackgerrit | Alexander Noskov proposed airship/promenade master: WIP https://review.opendev.org/652678 | 19:09 |
| *** graysonh has quit IRC | 19:16 | |
| AlexNoskov | Hi guys, please review/submit the following PS https://review.opendev.org/663812 | 19:20 |
| evgenyl | AlexNoskov: You can also ask for reviews related to openstack-helm on their #openstack-helm channel. | 19:22 |
| AlexNoskov | thx, will do | 19:23 |
| openstackgerrit | Ian Pittwood proposed airship/spyglass-plugin-xls master: Add unit tests for xls plugin CLI https://review.opendev.org/663142 | 19:24 |
| openstackgerrit | Stas Egorov proposed airship/airship-in-a-bottle master: [wip]: proxy config https://review.opendev.org/665460 | 19:28 |
| openstackgerrit | Stas Egorov proposed airship/airship-in-a-bottle master: [wip]: qemu https://review.opendev.org/665461 | 19:29 |
| openstackgerrit | Merged airship/spyglass-plugin-xls master: Implement PBR for XLS plugin https://review.opendev.org/664701 | 20:23 |
| openstackgerrit | Merged airship/spyglass-plugin-xls master: Adds force option to manifest generation https://review.opendev.org/662034 | 20:24 |
| openstackgerrit | Merged airship/spyglass-plugin-xls master: Update plugin docs https://review.opendev.org/661841 | 20:28 |
| openstackgerrit | Ian Pittwood proposed airship/spyglass-plugin-xls master: Fixes issues with tox https://review.opendev.org/662020 | 20:30 |
| openstackgerrit | Ian Pittwood proposed airship/spyglass-plugin-xls master: Fixes issues with tox https://review.opendev.org/662020 | 20:32 |
| openstackgerrit | Ian Pittwood proposed airship/spyglass-plugin-xls master: Add unit tests for xls plugin CLI https://review.opendev.org/663142 | 20:32 |
| *** kranthikirang has quit IRC | 20:35 | |
| openstackgerrit | Merged airship/promenade master: Haproxy: Add pod/container security context https://review.opendev.org/651574 | 20:38 |
| openstackgerrit | Ian Pittwood proposed airship/shipyard master: Update PyYAML to match other projects https://review.opendev.org/665472 | 20:45 |
| openstackgerrit | Ian Pittwood proposed airship/pegleg master: [WIP] Implement Pipenv dependency management https://review.opendev.org/664970 | 21:00 |
| openstackgerrit | Merged airship/treasuremap master: Remove pegleg clone from Zuul lint jobs https://review.opendev.org/665190 | 21:05 |
| *** howell has quit IRC | 21:34 | |
| openstackgerrit | Merged airship/promenade master: ETCD: Add pod/container security context https://review.opendev.org/662334 | 21:40 |
| openstackgerrit | Alexander Noskov proposed airship/promenade master: WIP https://review.opendev.org/652678 | 22:33 |
| openstackgerrit | Sean Eagan proposed airship/armada master: [WIP]: Prevent tiller from leaving releases in pending status https://review.opendev.org/665485 | 22:39 |
| openstackgerrit | Sean Eagan proposed airship/armada master: Prevent tiller from leaving releases in pending status https://review.opendev.org/665485 | 22:41 |
| *** sthussey has quit IRC | 23:47 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
