| *** fkautz_ has joined #openstack-kuryr | 00:37 | |
| *** fkautz_ is now known as fkautz | 00:38 | |
| *** fkautz has quit IRC | 00:38 | |
| *** fkautz has joined #openstack-kuryr | 00:38 | |
| *** yamamoto has joined #openstack-kuryr | 00:47 | |
| *** yamamoto has quit IRC | 00:52 | |
| *** salv-orlando has joined #openstack-kuryr | 00:59 | |
| *** salv-orlando has quit IRC | 01:03 | |
| *** salv-orlando has joined #openstack-kuryr | 02:00 | |
| *** salv-orlando has quit IRC | 02:04 | |
| *** hongbin has joined #openstack-kuryr | 02:32 | |
| *** janki has joined #openstack-kuryr | 02:35 | |
| *** salv-orlando has joined #openstack-kuryr | 03:00 | |
| *** maysamacedos has quit IRC | 03:02 | |
| *** salv-orlando has quit IRC | 03:05 | |
| *** jchhatbar has joined #openstack-kuryr | 03:19 | |
| *** janki has quit IRC | 03:22 | |
| *** salv-orlando has joined #openstack-kuryr | 04:01 | |
| *** salv-orlando has quit IRC | 04:05 | |
| *** gcheresh has joined #openstack-kuryr | 04:37 | |
| *** hongbin has quit IRC | 04:38 | |
| *** hongbin has joined #openstack-kuryr | 04:46 | |
| *** hongbin has quit IRC | 04:51 | |
| *** salv-orlando has joined #openstack-kuryr | 05:02 | |
| *** salv-orlando has quit IRC | 05:05 | |
| *** salv-orlando has joined #openstack-kuryr | 05:05 | |
| *** janonymous has joined #openstack-kuryr | 05:13 | |
| *** yboaron has joined #openstack-kuryr | 05:20 | |
| *** openstackgerrit has quit IRC | 05:48 | |
| *** caowei has quit IRC | 05:56 | |
| *** caowei has joined #openstack-kuryr | 05:58 | |
| *** caowei has quit IRC | 06:02 | |
| *** threestrands has joined #openstack-kuryr | 06:05 | |
| *** threestrands has quit IRC | 06:05 | |
| *** threestrands has joined #openstack-kuryr | 06:05 | |
| *** salv-orlando has quit IRC | 06:35 | |
| *** salv-orlando has joined #openstack-kuryr | 06:35 | |
| *** salv-orlando has quit IRC | 06:39 | |
| *** salv-orlando has joined #openstack-kuryr | 06:42 | |
| *** yamamoto has joined #openstack-kuryr | 07:05 | |
| *** yboaron has quit IRC | 07:14 | |
| *** pliu has joined #openstack-kuryr | 07:14 | |
| pliu | I got error "No 'kuryr_kubernetes.vif_translators' driver found" from kuryr-controller. Did I miss anything in the config file? | 07:16 |
|---|---|---|
| *** gcheresh has quit IRC | 07:17 | |
| celebdor | ltomasbo: ^^ | 07:19 |
| ltomasbo | celebdor, pliu: any more context? | 07:21 |
| dmellado | celebdor: your patch is in worse shape than cifuentes future in polytics | 07:21 |
| dmellado | could you take a look | 07:21 |
| ltomasbo | what driver are you trying to use, lbaas or octavia, kubernetes or openshift, ... | 07:21 |
| dmellado | maybe he can just share his local.conf | 07:21 |
| celebdor | yes, yes. I'm aware. It's just that I started doing the ansible part and everytime I touch ansible I get side tracked | 07:21 |
| * celebdor does not like coding for deployment tools | 07:22 | |
| pliu | ltomasbo, I haven't install octavia yet. Just trying to start a pod. Can kuryr-kubernetes work without octavia if I don't use service? | 07:23 |
| ltomasbo | pliu, yes, are you trying with devstack? with the default local.conf? | 07:25 |
| ltomasbo | pliu, also need to know if you run it containerized or not, daemonized or not, ... | 07:25 |
| pliu | ltomasbo, I'm trying to get it work on with RDO. The kuryr_kubernetes runs as a daemon on controller nodes | 07:26 |
| dmellado | pliu: you'd need either lbassv2 or octavia | 07:27 |
| ltomasbo | so, probably you are missing information at kuryr.conf, regarding the drivers you want to use | 07:28 |
| pliu | ltomasbo, so even without using k8s's service, I still need to setup lbassv2 or octavia? | 07:29 |
| dulek | pliu: Is it possible you simply don't have os-vif library installed? | 07:29 |
| ltomasbo | pliu, depend on how you configure it, you may not need any lbaas driver | 07:29 |
| dmellado | ltomasbo: yeah, that could be as well | 07:30 |
| dmellado | but that'd be the default usecase | 07:30 |
| pliu | dulek, I have os-vif installed. | 07:31 |
| pliu | which config parameter I should set to choose the vif translator dirver? | 07:32 |
| dulek | pliu: None. :D | 07:32 |
| dulek | pliu: It's chosen based on VIF provided by VIF driver. | 07:32 |
| dulek | pliu: So let's check what you have set as pod_vif_driver in [DEFAULT] section. | 07:33 |
| dulek | pliu: Oh sorry, it's [kubernetes] section. | 07:33 |
| * dulek needs more coffee. | 07:33 | |
| pliu | dulek, pod_vif_driver = neutron-vif | 07:36 |
| dmellado | pliu: that's ok | 07:37 |
| dulek | pliu: Can you list neutron ports and give us `openstack port show` of the one created by Kuryr? | 07:39 |
| pliu | +-----------------------+-------------------------------------------------------------------------------------+ | 07:42 |
| pliu | | Field | Value | | 07:42 |
| pliu | +-----------------------+-------------------------------------------------------------------------------------+ | 07:42 |
| pliu | | admin_state_up | True | | 07:42 |
| pliu | | allowed_address_pairs | | | 07:42 |
| pliu | | binding:host_id | overcloud-novacompute-1 | | 07:42 |
| pliu | | binding:profile | {} | | 07:42 |
| pliu | | binding:vif_details | {} | | 07:42 |
| pliu | | binding:vif_type | binding_failed | | 07:42 |
| pliu | | binding:vnic_type | normal | | 07:42 |
| pliu | | created_at | 2018-04-12T06:38:56Z | | 07:42 |
| *** openstackgerrit has joined #openstack-kuryr | 07:42 | |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Run OVN job containerized and daemonized https://review.openstack.org/549289 | 07:42 |
| pliu | | description | | | 07:42 |
| pliu | | device_id | cf23f36a-3e0a-11e8-aca7-52540049ded8 | | 07:42 |
| pliu | | device_owner | compute:kuryr | | 07:42 |
| pliu | | extra_dhcp_opts | | | 07:42 |
| pliu | | fixed_ips | {"subnet_id": "c6ac3c89-97ce-4ca8-b393-67bbae2ac58a", "ip_address": "192.168.0.41"} | | 07:42 |
| pliu | | id | aa3b06d8-6114-4fee-91c7-709c47be462d | | 07:42 |
| pliu | | mac_address | fa:16:3e:18:89:c2 | | 07:42 |
| pliu | | name | kuryr-poc | | 07:42 |
| pliu | | network_id | 7f5819b7-f75c-4e1b-9cef-cfebe28e7409 | | 07:42 |
| pliu | | port_security_enabled | True | | 07:42 |
| pliu | | project_id | dc5ec8f33a2f4e17b8f964ae8c7bc9e1 | | 07:42 |
| pliu | | qos_policy_id | | | 07:42 |
| pliu | | revision_number | 7 | | 07:42 |
| pliu | | security_groups | 16e7131e-7b80-494a-97ab-6e2963badb00 | | 07:42 |
| pliu | | status | DOWN | | 07:42 |
| pliu | | tags | | | 07:42 |
| pliu | | tenant_id | dc5ec8f33a2f4e17b8f964ae8c7bc9e1 | | 07:42 |
| pliu | | updated_at | 2018-04-12T06:38:57Z | | 07:42 |
| pliu | +-----------------------+-------------------------------------------------------------------------------------+ | 07:43 |
| pliu | I'm doing the container on baremetal scenario | 07:43 |
| dulek | pliu: Well, we cannot really do much with "binding_failed" VIF type. :P | 07:46 |
| dulek | pliu: Check out Neutron logs, there must have been a failure there. | 07:46 |
| dulek | pliu: https://github.com/openstack/neutron/blob/a51271d760b3a00f7648374d85fafb3d122a4985/doc/source/contributor/internals/live_migration.rst#potential-error-cases-related-to-networking-2 | 07:47 |
| dulek | I'm not sure if this is related. | 07:47 |
| dulek | celebdor, ltomasbo, dmellado, irenab: How do we handle that issue? Should we catch VIF_TYPE_BINDING_FAILED before translating and raise more specific exception? | 07:49 |
| *** gcheresh has joined #openstack-kuryr | 07:50 | |
| dulek | pliu: Can you file a bug on Launchpad? Even though it's not Kuryr's fault we should handle this better. | 07:50 |
| dmellado | dulek: probably yeah | 07:50 |
| dmellado | +1 | 07:50 |
| celebdor | dulek: we should handle it somehow, yes | 07:51 |
| irenab | dulek, what is the issue? | 07:51 |
| pliu | 2018-04-12 06:38:57.626 28 ERROR neutron.plugins.ml2.managers [req-650281cd-f905-45ac-ae3e-3d570c74ad22 365c237b49634df9a6e0e49d5dc28d53 dc5ec8f33a2f4e17b8f964ae8c7bc9e1 - default default] Failed to bind port aa3b06d8-6114-4fee-91c7-709c47be462d on host overcloud-novacompute-1 for vnic_type normal using segments [{'network_id': '7f5819b7-f75c-4e1b-9cef-cfebe28e7409', 'segmentation_id': 27, 'physical_network': None, 'id': '717be8a3-7978- | 07:51 |
| pliu | 4e2e-88b6-5ba407a2b58a', 'network_type': u'vxlan'}] | 07:51 |
| pliu | 2018-04-12 06:38:57.627 28 INFO neutron.plugins.ml2.plugin [req-650281cd-f905-45ac-ae3e-3d570c74ad22 365c237b49634df9a6e0e49d5dc28d53 dc5ec8f33a2f4e17b8f964ae8c7bc9e1 - default default] Attempt 2 to bind port aa3b06d8-6114-4fee-91c7-709c47be462d | 07:51 |
| pliu | I get this from neutron server log | 07:52 |
| dulek | irenab: Neutron can signal an issue with the binding of created port by setting its type to VIF_TYPE_BINDING_FAILED. Then in Kuryr we get the exception mentioned originally by pliu. | 07:52 |
| irenab | dulek, pliu I think the log on kuryr side can bebetter to reflect the issue of unsupported binding type (failed is just case of the unsupported), the rest should come from the neutron. | 07:54 |
| dulek | pliu: Sorry, I have no idea what causes that. But I guess more specific failure can be found in L2 agent logs. | 07:54 |
| pliu | dulek, I am not sure if it is bug. I install and config kuryr-kubernetes manually. Maybe just a config error by me? | 07:55 |
| irenab | pliu, it looks like there is a segmented network on the neutron side. Is this what you wanted? | 07:55 |
| dulek | pliu: It's definitely Neutron failure. | 07:55 |
| dulek | pliu: Or Neutron misconfiguration. But it doesn't look like Kuryr's fault. | 07:55 |
| dulek | irenab: Yep, I was just thinking of logging that this issue most likely comes from Neutron, not Kuryr. | 07:56 |
| irenab | Agree with dulek . Since there is no valid vif-type returned from neutron, there is no much kuryr can do | 07:56 |
| pliu | irenab, the network I'm using for kuryr is a vxlan network which create by me. Is that the issue? | 07:57 |
| irenab | pliu, I was curious about 'using segments' in the log message you posted | 07:57 |
| irenab | anything in the l2_agent log? | 07:58 |
| *** gcheresh_ has joined #openstack-kuryr | 07:59 | |
| pliu | +---------------------------+--------------------------------------+| Field | Value | | 07:59 |
| pliu | +---------------------------+--------------------------------------+ | 07:59 |
| pliu | | admin_state_up | True | | 07:59 |
| pliu | | availability_zone_hints | | | 07:59 |
| pliu | | availability_zones | nova | | 07:59 |
| pliu | | created_at | 2018-04-04T15:38:55Z | | 07:59 |
| pliu | | description | | | 07:59 |
| dmellado | pliu: do you have an answers file from packsack around? | 07:59 |
| pliu | | id | 7f5819b7-f75c-4e1b-9cef-cfebe28e7409 | | 07:59 |
| pliu | | ipv4_address_scope | | | 07:59 |
| pliu | | ipv6_address_scope | | | 07:59 |
| dmellado | also, please do use pastebin | 07:59 |
| pliu | | mtu | 1450 | | 07:59 |
| pliu | | name | internal | | 07:59 |
| pliu | | port_security_enabled | True | | 07:59 |
| pliu | | project_id | dc5ec8f33a2f4e17b8f964ae8c7bc9e1 | | 07:59 |
| pliu | | provider:network_type | vxlan | | 07:59 |
| pliu | | provider:physical_network | | | 07:59 |
| dmellado | otherwise it's just flood on the channel | 07:59 |
| *** gcheresh has quit IRC | 07:59 | |
| pliu | | provider:segmentation_id | 27 | | 07:59 |
| pliu | | qos_policy_id | | | 07:59 |
| pliu | | revision_number | 4 | | 07:59 |
| pliu | | router:external | False | | 07:59 |
| pliu | | shared | False | | 07:59 |
| pliu | | status | ACTIVE | | 07:59 |
| pliu | | subnets | c6ac3c89-97ce-4ca8-b393-67bbae2ac58a | | 07:59 |
| pliu | | tags | | | 07:59 |
| pliu | | tenant_id | dc5ec8f33a2f4e17b8f964ae8c7bc9e1 | | 08:00 |
| dmellado | pliu: http://paste.openstack.org/ | 08:00 |
| pliu | | updated_at | 2018-04-04T15:39:15Z | | 08:00 |
| pliu | +---------------------------+--------------------------------------+ | 08:00 |
| pliu | this is the network info | 08:00 |
| pliu | https://pastebin.com/Qvqzprek | 08:00 |
| irenab | pliu, network seems fine. Try to check the l2 agent logs, maybe some mappings are missing | 08:02 |
| ltomasbo | pliu, how did you created that port? was kuryr creating it for you? | 08:05 |
| pliu | I didn't find any error in ovs agent log on compute node | 08:06 |
| pliu | Yes, that port was created by kuryr | 08:06 |
| ltomasbo | ok, celebdor did you remember for the defconf demo, we had to do some workaround because the port was going to binding failure? | 08:07 |
| pliu | I tried to boot a vm on that compute node, it works as expect. | 08:07 |
| dulek | pliu: Hm, had you checked the logs of kubelet? Maybe for some reason it's Kuryr-CNI breaking something? | 08:07 |
| ltomasbo | that was because the agent was down, right? | 08:07 |
| ltomasbo | pliu, can you check the status of the neutron agents? | 08:07 |
| pliu | in kubelet log, I can see the kuryr-cni get timeout when trying to bind if | 08:08 |
| *** yboaron has joined #openstack-kuryr | 08:10 | |
| ltomasbo | pliu, can you do 'openstack network agent list'? | 08:10 |
| pliu | https://pastebin.com/GR3JhJ1c all work are on overcloud-novacompute-1. | 08:12 |
| *** pcaruana has joined #openstack-kuryr | 08:12 | |
| *** celebdor1 has joined #openstack-kuryr | 08:13 | |
| *** celebdor1 is now known as apuimedo | 08:13 | |
| apuimedo | I had gnome broken | 08:13 |
| ltomasbo | yep, I see ovs is down on overcoud-novacompute-0 | 08:13 |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: Add tarballed amphora. https://review.openstack.org/560313 | 08:13 |
| apuimedo | I've had to move to cinnamon temporarily :/ | 08:13 |
| apuimedo | dmellado: is this patch for real?! | 08:14 |
| ltomasbo | pliu, perhaps a restart of the agent on overcloud-novacompute-1 will help | 08:14 |
| dmellado | apuimedo: fixing botched jobs from octavia, but yeah | 08:14 |
| dmellado | xD | 08:14 |
| dmellado | so we'd be able to skip the dib bits | 08:14 |
| apuimedo | dmellado: dib? | 08:15 |
| dmellado | disk image builder | 08:16 |
| pliu | ltomasbo, I think the ovs-agent is ok. Since I just boot a nova vm on that compute. It works, and port is attached | 08:16 |
| *** yamamoto has quit IRC | 08:17 | |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: Add tarballed amphora. https://review.openstack.org/560313 | 08:23 |
| apuimedo | ah | 08:24 |
| *** salv-orlando has quit IRC | 08:30 | |
| *** salv-orlando has joined #openstack-kuryr | 08:31 | |
| *** yamamoto has joined #openstack-kuryr | 08:32 | |
| *** salv-orlando has quit IRC | 08:35 | |
| *** yamamoto has quit IRC | 08:36 | |
| *** shadower has quit IRC | 08:41 | |
| *** shadower has joined #openstack-kuryr | 08:41 | |
| *** garyloug has joined #openstack-kuryr | 08:45 | |
| *** natanbro has joined #openstack-kuryr | 09:25 | |
| *** salv-orlando has joined #openstack-kuryr | 09:31 | |
| *** salv-orlando has quit IRC | 09:37 | |
| *** salv-orlando has joined #openstack-kuryr | 09:39 | |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: Add tarballed amphora. https://review.openstack.org/560313 | 09:43 |
| *** yamamoto has joined #openstack-kuryr | 09:46 | |
| *** yamamoto has quit IRC | 09:48 | |
| *** yamamoto has joined #openstack-kuryr | 09:49 | |
| *** yamamoto has quit IRC | 09:52 | |
| *** yamamoto has joined #openstack-kuryr | 09:54 | |
| apuimedo | I just fixed the unit test for the octavia client fix | 09:55 |
| apuimedo | I'm adding one extra to cover the new code | 09:55 |
| *** yamamoto has quit IRC | 10:00 | |
| *** yamamoto has joined #openstack-kuryr | 10:01 | |
| openstackgerrit | Antoni Segura Puimedon proposed openstack/kuryr-kubernetes master: lbaasv2: Fix for deployments missing the proxy https://review.openstack.org/560433 | 10:11 |
| apuimedo | dmellado: dulek: ltomasbo: yboaron: irenab: ^^ | 10:15 |
| irenab | apuimedo, checking | 10:16 |
| irenab | apuimedo, I do not understadnwhat the comment https://review.openstack.org/#/c/560433/3/kuryr_kubernetes/clients.py line 56 means | 10:17 |
| *** threestrands has quit IRC | 10:24 | |
| openstackgerrit | Antoni Segura Puimedon proposed openstack/kuryr-kubernetes master: lbaasv2: Fix for deployments missing the proxy https://review.openstack.org/560433 | 10:32 |
| apuimedo | irenab: it means that since the APIs are compatible, we just use neutronclient to talk to it | 10:32 |
| apuimedo | but since neutronclient can only have one endpoint, we create an extra instance | 10:33 |
| apuimedo | so we have one instance to talk to neutron and one to talk to lbaas | 10:33 |
| * apuimedo going to cook rice | 10:33 | |
| dmellado | apuimedo: I left also some comments there | 10:33 |
| dmellado | please check an address PS3 ones | 10:35 |
| dmellado | and *not* playstation xD | 10:35 |
| irenab | apuimedo, got it, now it is clear, maybe worth to clarify the comment in the code. | 10:37 |
| irenab | dmellado :-) | 10:38 |
| dmellado | ;) | 10:39 |
| dmellado | btw irenab I got multinode gates working, pending on a dependency on devstack to merge | 10:40 |
| dmellado | once that you fix DF issues I'll also add a multinode gate for it | 10:40 |
| irenab | dmellado, the DF issue seems to be a bit fundamental change. I wonder if we can split the tests for pod to Vm connectivity and temporary disable failing test for DF | 10:44 |
| dmellado | irenab: yeah, that won't be an issue, we can just add a blacklist for DF | 10:44 |
| dmellado | irenab: I'll put an example on how to do this and add you as reviewer so you can take it over and hande this as you might want | 10:45 |
| dmellado | hande/handle | 10:45 |
| irenab | dmellado, thanks | 10:45 |
| *** yamamoto has quit IRC | 10:56 | |
| *** yamamoto has joined #openstack-kuryr | 11:02 | |
| *** yamamoto has quit IRC | 11:03 | |
| *** yamamoto has joined #openstack-kuryr | 11:08 | |
| *** yamamoto has quit IRC | 11:11 | |
| *** yamamoto has joined #openstack-kuryr | 11:11 | |
| dulek | apuimedo: lbaasv2 job failed. | 11:21 |
| dulek | apuimedo: I'm investigating. | 11:21 |
| dulek | apuimedo, dmellado: http://logs.openstack.org/33/560433/4/check/kuryr-kubernetes-tempest-daemon-lbaasv2/080db9a/job-output.txt.gz#_2018-04-12_11_00_57_473298 | 11:22 |
| dulek | That's quite odd. | 11:22 |
| dulek | Aww, crap. K8s client update broke us again? | 11:25 |
| dulek | Yep, they've just released. AWESOME! | 11:26 |
| dulek | I'm capping it. | 11:28 |
| *** yamamoto has quit IRC | 11:33 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-tempest-plugin master: Cap kubernetes Python package to <6.0.0 https://review.openstack.org/560867 | 11:36 |
| dulek | apuimedo, dmellado, irenab: Let's see if that solves the issue and merge it to unblock. | 11:36 |
| dulek | I've capped instead of fixing it as I'm not sure how this will behave on stable branch. | 11:36 |
| dulek | And global-requirements has >=4.0.0, so 5.0.0 fits. | 11:38 |
| *** maysamacedos has joined #openstack-kuryr | 11:52 | |
| dmellado | dulek: I'm so happy with your performance xD | 11:54 |
| dmellado | I noticed that and I was about to tackle that after lunch | 11:54 |
| dmellado | but I come back from it and it got solved xD xD xD | 11:55 |
| dulek | dmellado: Frustration tends to fuel me sometimes. Like why every release of that client breaks up?! | 11:55 |
| dulek | s/up/us | 11:56 |
| apuimedo | dulek: because it's trendy | 11:58 |
| apuimedo | trendiness is very strongly correlated with not giving a fuck about bc | 11:59 |
| *** yboaron_ has joined #openstack-kuryr | 12:00 | |
| *** yboaron has quit IRC | 12:04 | |
| *** atoth has joined #openstack-kuryr | 12:05 | |
| *** yamamoto has joined #openstack-kuryr | 12:30 | |
| *** yamamoto has quit IRC | 12:35 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-tempest-plugin master: Add V1DeleteOptions when deleting Service https://review.openstack.org/560867 | 12:39 |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-tempest-plugin master: Apply cap from upper reqs https://review.openstack.org/560885 | 12:43 |
| dmellado | dulek: celebdor ltomasbo https://review.openstack.org/560885 | 12:43 |
| dmellado | please let's get this merged asap to unlock the gates | 12:43 |
| dulek | dmellado: Oh, nice! | 12:43 |
| dmellado | oddly it wasn't applying the upper req limits | 12:44 |
| dulek | dmellado: Good idea, but once upper-requirements.txt are changed we're broken again… | 12:45 |
| dmellado | not really | 12:45 |
| dmellado | upper reqs is mapped to 5 | 12:45 |
| dmellado | and I'm pretty sure they'll skip the version | 12:45 |
| dmellado | if that changes it in that way | 12:45 |
| dulek | dmellado: Yes, but uc might be raised. | 12:45 |
| dmellado | dulek: when that comes, we'll address that either in our reqs or we fix the change | 12:46 |
| dmellado | uc will in any case just be raised for the release | 12:46 |
| dmellado | which we'll address as well | 12:46 |
| dulek | dmellado: Okay. Then I'm -W on https://review.openstack.org/560867. | 12:46 |
| dmellado | dulek: great, thanks! | 12:47 |
| *** yamamoto has joined #openstack-kuryr | 12:47 | |
| *** yamamoto has quit IRC | 12:49 | |
| garyloug | Hi folks, I've posted some comments on this binding driver https://review.openstack.org/#/c/559363/1/kuryr_kubernetes/cni/binding/dpdk.py If you have time I'd like to get some input on what you think is the best way to do this. | 12:51 |
| irenab | dmellado, dulek maybe worth to raise the broken BC to the attention of the k8s client team? | 12:52 |
| dulek | irenab: They'll probably say that they're changing major version, so that's expected. | 12:53 |
| dmellado | dulek: irenab even worse, IIRC they're beta | 12:53 |
| dulek | But the only releases they do are raising major. | 12:53 |
| dulek | :D | 12:53 |
| dmellado | so they advise that something like that might happen | 12:53 |
| dulek | Oh my. | 12:53 |
| dmellado | in any case, and that said | 12:53 |
| dmellado | the change looked minimal | 12:53 |
| dmellado | like def(foo, bar) to def(foo, bar, oh_my) | 12:54 |
| dmellado | but let's keep the uc constraints for now to unlock the gate and I'll take a look afterwards | 12:54 |
| *** maysamacedos has quit IRC | 12:54 | |
| dulek | They're autogenerating that based on Swagger, I think. | 12:54 |
| dmellado | yeah, exactly | 12:54 |
| irenab | never though that autogenerating comes with 'do not bother to keep bc' ... | 12:55 |
| dmellado | heh, lol | 12:55 |
| *** yamamoto has joined #openstack-kuryr | 13:01 | |
| *** yamamoto has quit IRC | 13:01 | |
| apuimedo | thanks garyloug | 13:02 |
| apuimedo | I'll take a look | 13:02 |
| *** yamamoto has joined #openstack-kuryr | 13:05 | |
| *** jchhatbar is now known as janki | 13:07 | |
| *** yamamoto has quit IRC | 13:09 | |
| dmellado | yep, me too | 13:18 |
| dmellado | yboaron_: ping, could you please also check https://review.openstack.org/559363 | 13:18 |
| dmellado | ? | 13:18 |
| yboaron_ | dmellado, sure , in few minutes | 13:18 |
| dmellado | yboaron_: whenever you've some time, no worries ;) | 13:19 |
| *** yamamoto has joined #openstack-kuryr | 13:21 | |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: Add tarballed amphora. https://review.openstack.org/560313 | 13:24 |
| *** yamamoto has quit IRC | 13:24 | |
| *** janonymous has quit IRC | 13:25 | |
| *** yboaron_ has quit IRC | 13:30 | |
| *** yboaron_ has joined #openstack-kuryr | 13:31 | |
| *** yamamoto has joined #openstack-kuryr | 13:43 | |
| garyloug | Thanks guys | 13:44 |
| *** yamamoto has quit IRC | 13:44 | |
| *** dangtrinhnt has quit IRC | 13:45 | |
| *** dangtrinhnt has joined #openstack-kuryr | 13:46 | |
| *** yamamoto has joined #openstack-kuryr | 13:48 | |
| *** yamamoto has quit IRC | 13:49 | |
| *** yamamoto has joined #openstack-kuryr | 13:49 | |
| *** hongbin has joined #openstack-kuryr | 14:13 | |
| dulek | dmellado: Your fix is red in the gate, seems like wrong package is still installed. | 14:17 |
| *** yboaron_ has quit IRC | 14:22 | |
| openstackgerrit | Genadi Chereshnya proposed openstack/kuryr-tempest-plugin master: Testing connectivity from VM to service and vice versa https://review.openstack.org/560926 | 14:26 |
| dmellado | dulek: | 14:29 |
| dmellado | I've just seen it | 14:29 |
| dmellado | oddly | 14:29 |
| dmellado | it's on kubernetes 5 *inside* the tox env | 14:29 |
| dmellado | but kubernetes 6 outside | 14:29 |
| dmellado | makes totally no sense and I'm not being able to get a match on the pip freeze | 14:30 |
| dmellado | xD | 14:30 |
| dulek | dmellado: Well, it doesn't ring a bell for me. :( | 14:30 |
| dmellado | I'm tempted to just fix the issue on the kubernetes library, tbh | 14:30 |
| dulek | dmellado: I'm rechecking my patch - Octavia timed out but that's probably unrelated. | 14:30 |
| dmellado | hm, I think I'm starting to know what's going on | 14:31 |
| dulek | dmellado: You would need a release of it, easier to fix it on our side. | 14:31 |
| dmellado | let me check a second | 14:31 |
| dmellado | dulek: I mean | 14:31 |
| dmellado | fix our code onthe tempest plugin to match kuberntes 6 | 14:31 |
| dulek | dmellado: https://review.openstack.org/#/c/560867/ - that's done here. :D | 14:31 |
| dulek | I'm just rechecking it as Octavia timed out - seems it's unrelated. | 14:32 |
| *** garyloug has quit IRC | 14:32 | |
| *** garyloug has joined #openstack-kuryr | 14:32 | |
| dmellado | dulek: -1 on that patch | 14:32 |
| dmellado | you're not fixing the callback | 14:32 |
| dmellado | xD | 14:32 |
| dulek | Callback? | 14:33 |
| dmellado | yep | 14:33 |
| dmellado | let me comment it there | 14:33 |
| dmellado | it would work but that's not the best ever possible approach | 14:33 |
| dulek | dmellado: Okay, please comment with explanation and I'll upload a new one. | 14:34 |
| dulek | gcheresh_: Just in case - gate is pretty much broken due to kubernetes lib release. | 14:34 |
| dulek | gcheresh_: We're working on that. | 14:35 |
| dmellado | dulek: see my comments there | 14:36 |
| dmellado | if you've any doubt let me know | 14:36 |
| dmellado | my patch works inside the venv but thare's some odd thing being done there | 14:37 |
| dmellado | so, when they trigger the plugins they do a tox -eall-plugin which installs a system-wide venv | 14:37 |
| dmellado | so the outside kubernetes 6 takes over the one inside the plugin | 14:38 |
| dmellado | odd in any case | 14:38 |
| dulek | dmellado: Wait, what? | 14:38 |
| dulek | dmellado: It's delete body. | 14:38 |
| dulek | dmellado: Not a service. | 14:38 |
| dulek | dmellado: V1DeleteOptions, not V1Service. | 14:38 |
| * dmellado facepalms | 14:38 | |
| dmellado | nvm | 14:38 |
| dmellado | I was thinking about service body | 14:39 |
| dulek | dmellado: Yeah, me too until I've looked into the code. | 14:39 |
| dmellado | do rebase the tox thing in any case and i'll approve it | 14:39 |
| dulek | dmellado: Pretty odd choice of a name for delete options. | 14:39 |
| gcheresh_ | dulek: dmellado The patch I uploaded has problems so if you can take a look what am I missing it will be great | 14:40 |
| dmellado | dulek: in any case | 14:40 |
| dmellado | there's something odd on the infra side | 14:40 |
| dmellado | as whatever installs kubernetes python should also match upper reqs | 14:41 |
| dulek | gcheresh_: You're referring to my messages from 16:34? | 14:41 |
| dulek | dmellado: It's Tempest. Maybe it's not using them. | 14:41 |
| gcheresh_ | dulek: No I think there is some problems on connectivity in devstack as I can reach from VM the pods of the service but can't reach the service itself | 14:42 |
| dulek | dmellado: You want me to add the body parameter to delete_service()? I can do it but not sure if it's too useful. If anyone needs it he'll just extend delete_service(). | 14:42 |
| dmellado | dulek: no, it actually should | 14:43 |
| dmellado | I've just checked their tox.ini | 14:43 |
| dmellado | dulek: nope, no need if it's another body | 14:43 |
| dmellado | disregard that | 14:43 |
| dmellado | just rebase the tox upper reqs for future usage | 14:43 |
| dulek | gcheresh_: Okay, I'll take a look a bit later on or tomorrow. | 14:43 |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-tempest-plugin master: Apply cap from upper reqs https://review.openstack.org/560885 | 14:44 |
| dulek | dmellado: Okay, let's see how this will work in the gate. | 14:44 |
| * dulek leaves for ~30 minutes. | 14:44 | |
| dmellado | dulek: ack, I'll be leaving in around that time too | 14:45 |
| dmellado | so I'll leave some comments on the patch | 14:45 |
| *** gcheresh_ has quit IRC | 14:47 | |
| dmellado | dulek: that kubernetes 6 is around makes no sense at all | 14:52 |
| * dmellado sighs | 14:52 | |
| dmellado | dulek: in any case we need to be way on this | 14:55 |
| dmellado | as if we fix this in the proper way the gate would fail again with yout patch | 14:55 |
| *** natanbro has quit IRC | 15:00 | |
| *** gcheresh_ has joined #openstack-kuryr | 15:16 | |
| dulek | dmellado: It'll not, I'm passing this `body` as kwarg and there's **kwargs in definiton. | 15:21 |
| dmellado | dulek: in any case I don't think that's a proper fix | 15:23 |
| dmellado | I commented on your patch | 15:23 |
| dmellado | imagine that python kubernetes release 7 tomorrow | 15:23 |
| dmellado | and they add a madatory tuple parameter | 15:23 |
| dmellado | like (apuimedo , lazy) | 15:24 |
| dmellado | we'd get broken at that same moment | 15:24 |
| dmellado | we need to dig out why the version over the reqs is being installed | 15:24 |
| dmellado | even if eventually accept that patch as a partial fix | 15:24 |
| dulek | dmellado: Okay, can you point me to gate results which install wrong version? | 15:25 |
| dulek | dmellado: I can look were it's getting installed. | 15:25 |
| dmellado | dulek: I couldn't fetch that upstream but I'm redeploying and it's the only thing that make sense to me | 15:26 |
| dmellado | I'll be asking infra to freeze the machine if get the same results on my patch | 15:26 |
| dmellado | like, we know that by upper reqs we're installing 5 and that requires no change on the method | 15:26 |
| dmellado | even with that, it complains | 15:26 |
| dmellado | and I've seen on my machine, now being redeployed that | 15:26 |
| dulek | Damn, we've probably lost it due to my rebase. Awww… | 15:27 |
| dmellado | it installed kubernetes 6 on the system-site | 15:27 |
| dmellado | yeah, I'm waiting for those | 15:27 |
| dmellado | but 5 inside the tox env | 15:27 |
| dmellado | so weird | 15:27 |
| dulek | Okay, found it - http://logs.openstack.org/85/560885/1/check/kuryr-kubernetes-tempest-daemon-lbaasv2/3bc2f26/ | 15:27 |
| dulek | Now let's just see where it was installed. | 15:27 |
| dmellado | lol, I was being overly optimistic and thought you had found the place where it was installed | 15:28 |
| dmellado | xD | 15:28 |
| dulek | dmellado: There's 6.0.0 in venv as well. | 15:28 |
| dulek | http://logs.openstack.org/85/560885/1/check/kuryr-kubernetes-tempest-daemon-lbaasv2/3bc2f26/job-output.txt.gz#_2018-04-12_13_11_12_360399 | 15:28 |
| dmellado | dulek: where did you find it? | 15:29 |
| dmellado | makes no sense | 15:29 |
| dulek | dmellado: It does make sense. | 15:30 |
| dulek | dmellado: http://logs.openstack.org/85/560885/1/check/kuryr-kubernetes-tempest-daemon-lbaasv2/3bc2f26/job-output.txt.gz#_2018-04-12_13_11_07_884256 | 15:30 |
| dulek | dmellado: It's installing kuryr-tempest-plugin into the venv. | 15:30 |
| dulek | dmellado: And that's going without upper-constraints. At least it looks like it. | 15:30 |
| dulek | dmellado: pip install ../kuryr-tempest-plugin | 15:30 |
| dulek | That's it. | 15:30 |
| dmellado | 2018-04-12 13:11:00.250597 | controller | ++ lib/tempest:install_tempest_plugins:649 : tox -evenv-tempest -- pip install /opt/stack/kuryr-tempest-plugin | 15:31 |
| dmellado | yep | 15:31 |
| dmellado | that's a bug | 15:31 |
| dmellado | in the tempest code | 15:31 |
| dmellado | awesome | 15:31 |
| dmellado | I wonder how anyone didn't notice | 15:31 |
| dulek | dmellado: https://github.com/openstack-dev/devstack/blob/master/lib/tempest#L645-L654 | 15:32 |
| dulek | dmellado: In DevStack. | 15:32 |
| dulek | dmellado: Fancy for a DevStack commit? ;) | 15:38 |
| dmellado | dulek: lol | 15:38 |
| dulek | dmellado: Looks like it's a trivial one. I can do it if you want. | 15:40 |
| dmellado | no worries, I'll just push it | 15:40 |
| dmellado | totally trivial | 15:40 |
| dmellado | just -c $REQUIREMENTS_DIR/upper | 15:40 |
| dmellado | on the install tempest plugins | 15:40 |
| dmellado | - tox -evenv-tempest -- pip install $TEMPEST_PLUGINS | 15:42 |
| dmellado | + tox -evenv-tempest -- pip install -c $REQUIREMENTS_DIR/upper-constraints.txt TEMPEST_PLUGINS | 15:42 |
| dmellado | tadaaa | 15:42 |
| dmellado | xD | 15:42 |
| dmellado | $TEMPEST_PLUGINS | 15:43 |
| dmellado | xD | 15:43 |
| dmellado | dulek: if you're around later could you pls watch the change in case it needs something more? | 15:43 |
| dulek | dmellado: Yup. Might be worth creating a Kuryr-kubernetes dummy commit with Depends-On on it. | 15:43 |
| dmellado | dulek: https://review.openstack.org/560946 | 15:44 |
| * dmellado flees away xD | 15:44 | |
| dulek | dmellado: I'll add that change. ;) | 15:44 |
| dulek | dmellado: But you're missing bug link! | 15:44 |
| *** gcheresh_ has quit IRC | 15:46 | |
| dmellado | dulek: I know :P | 15:46 |
| dmellado | amend it, pls xD | 15:46 |
| dmellado | we'll assign next one to juriarte and gcheresh | 15:47 |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Do not merge, testing DevStack fix https://review.openstack.org/560951 | 15:50 |
| dulek | apuimedo: Seen my comment on your patch? The Octavia gates are failing in a weird way. | 16:02 |
| dulek | apuimedo: It's worth trying to switch order of post-config and LIBS_FROM_GIT in the definition. | 16:03 |
| *** gcheresh_ has joined #openstack-kuryr | 16:15 | |
| dulek | apuimedo, ltomasbo: Do you think we can cleanup https://hub.docker.com/r/kuryr/cni/tags/ ? | 16:19 |
| ltomasbo | dulek, perhaps not all of them | 16:20 |
| ltomasbo | dulek, is latest already fixing the problem with os_vif and the pleg mitigation? | 16:21 |
| ltomasbo | I guess multipools, multipool-no-rivsep can be deleted | 16:21 |
| ltomasbo | not sure what a50bf74d is | 16:22 |
| dulek | ltomasbo: Uh… Definitely it's fixing os_vif (docker exec based CNI execution solves that). | 16:22 |
| dulek | ltomasbo: I don't remember PLEG mitigation? | 16:22 |
| ltomasbo | I believe the pleg thing was fixed upstream kubernetes, but perhaps apuimedo remembers that better | 16:22 |
| ltomasbo | if that so, I guess you can clean them all | 16:22 |
| dulek | ltomasbo: Okay. Meanwhile I want to push latest master. | 16:25 |
| dulek | ltomasbo: And it'll need this: https://github.com/openshift/openshift-ansible/pull/7889 | 16:26 |
| dulek | shadower: Can you take a look on this: https://github.com/openshift/openshift-ansible/pull/7889 ? | 16:26 |
| ltomasbo | dulek, we don't need the lock_path anymore? | 16:27 |
| ltomasbo | i thought it was needed for the cni to properly work | 16:27 |
| dulek | ltomasbo: https://github.com/openstack/kuryr-kubernetes/blob/206c158d37060142d7d80d1be0564be10af576e7/cni.Dockerfile#L26 | 16:27 |
| dulek | ltomasbo: Not with this. | 16:27 |
| ltomasbo | ahh, ok, you fix it on the other side | 16:28 |
| ltomasbo | ok then | 16:28 |
| ltomasbo | it is a bit of a pain to have to do that twice (kuryr-kubernetes and openshift-ansible) | 16:28 |
| dulek | Yup. | 16:29 |
| *** garyloug has quit IRC | 16:34 | |
| *** gcheresh_ has quit IRC | 16:46 | |
| *** yamamoto has quit IRC | 16:47 | |
| *** yamamoto has joined #openstack-kuryr | 16:58 | |
| *** yamamoto has quit IRC | 17:03 | |
| *** janki has quit IRC | 17:09 | |
| *** russellb has quit IRC | 17:13 | |
| *** leifmadsen has quit IRC | 17:13 | |
| apuimedo | ltomasbo: it should be | 17:14 |
| ltomasbo | apuimedo, great! thanks! | 17:15 |
| apuimedo | ltomasbo: but the fix IIRC was in Docker so... | 17:16 |
| apuimedo | it depends on the distro if we get it or not | 17:17 |
| openstackgerrit | Antoni Segura Puimedon proposed openstack/kuryr-kubernetes master: lbaasv2: Fix for deployments missing the proxy https://review.openstack.org/560433 | 17:17 |
| *** juriarte has quit IRC | 17:18 | |
| *** juriarte has joined #openstack-kuryr | 17:18 | |
| *** leifmadsen has joined #openstack-kuryr | 17:21 | |
| *** russellb has joined #openstack-kuryr | 17:21 | |
| *** gcheresh_ has joined #openstack-kuryr | 17:33 | |
| *** yamamoto has joined #openstack-kuryr | 18:04 | |
| *** pcaruana has quit IRC | 18:09 | |
| *** openstackgerrit has quit IRC | 18:19 | |
| *** gcheresh_ has quit IRC | 18:20 | |
| *** salv-orlando has quit IRC | 19:04 | |
| *** salv-orlando has joined #openstack-kuryr | 19:04 | |
| *** salv-orlando has quit IRC | 19:09 | |
| *** apuimedo has quit IRC | 19:09 | |
| *** yboaron_ has joined #openstack-kuryr | 19:17 | |
| *** aojea has joined #openstack-kuryr | 19:41 | |
| *** apuimedo has joined #openstack-kuryr | 20:03 | |
| *** lihi has quit IRC | 20:03 | |
| *** irenab has quit IRC | 20:04 | |
| *** snapiri has quit IRC | 20:04 | |
| *** oanson has quit IRC | 20:05 | |
| *** salv-orlando has joined #openstack-kuryr | 20:05 | |
| *** atoth has quit IRC | 20:06 | |
| *** salv-orlando has quit IRC | 20:09 | |
| *** salv-orlando has joined #openstack-kuryr | 20:09 | |
| *** snapiri has joined #openstack-kuryr | 20:11 | |
| *** irenab has joined #openstack-kuryr | 20:11 | |
| *** oanson has joined #openstack-kuryr | 20:12 | |
| *** lihi has joined #openstack-kuryr | 20:12 | |
| *** yamamoto has quit IRC | 20:21 | |
| *** yamamoto has joined #openstack-kuryr | 20:22 | |
| *** salv-orlando has quit IRC | 20:47 | |
| *** salv-orlando has joined #openstack-kuryr | 20:48 | |
| *** openstackgerrit has joined #openstack-kuryr | 21:19 | |
| openstackgerrit | Yossi Boaron proposed openstack/kuryr-kubernetes master: [WIP] Make handlers pluggable https://review.openstack.org/555735 | 21:19 |
| *** aojea has quit IRC | 21:26 | |
| *** aojea has joined #openstack-kuryr | 21:26 | |
| *** yboaron_ has quit IRC | 21:26 | |
| *** openstack has joined #openstack-kuryr | 21:28 | |
| *** ChanServ sets mode: +o openstack | 21:28 | |
| *** aojea has quit IRC | 21:56 | |
| *** salv-orlando has quit IRC | 22:18 | |
| *** salv-orlando has joined #openstack-kuryr | 22:19 | |
| *** salv-orlando has quit IRC | 22:23 | |
| openstackgerrit | Antoni Segura Puimedon proposed openstack/kuryr-kubernetes master: lbaasv2: Fix for deployments missing the proxy https://review.openstack.org/560433 | 22:24 |
| *** hongbin has quit IRC | 22:58 | |
| *** ripdogPJ0R9A has joined #openstack-kuryr | 23:40 | |
| *** ripdogPJ0R9A has quit IRC | 23:42 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!