Tuesday, 2020-12-01

*** nsegkos has joined #openstack00:00
*** nksegos has quit IRC00:01
*** nsegkos is now known as nksegos00:01
*** cyberworm54 has quit IRC00:15
*** cyberworm54 has joined #openstack00:16
*** nksegos has quit IRC00:32
*** macz_ has quit IRC00:40
*** jv has joined #openstack01:01
*** Yarboa has quit IRC01:03
*** Yarboa has joined #openstack01:05
*** Goneri has quit IRC01:30
*** Yarboa has quit IRC02:03
*** Yarboa has joined #openstack02:04
*** gavlee has quit IRC02:40
*** gavlee has joined #openstack02:44
*** Yarboa has quit IRC03:03
*** Yarboa has joined #openstack03:03
*** rcernin has quit IRC03:04
*** rcernin has joined #openstack03:18
*** rcernin has quit IRC03:21
*** rcernin has joined #openstack03:22
*** gudrutis has quit IRC03:31
*** gudrutis has joined #openstack03:31
*** Yarboa has quit IRC03:54
*** Yarboa has joined #openstack03:54
*** manuvakery_ has joined #openstack05:05
*** rockie has joined #openstack05:05
*** rvd has quit IRC05:08
*** rcernin has quit IRC05:22
*** zbsarashki has quit IRC05:29
*** rcernin has joined #openstack05:32
*** kmarc has quit IRC05:53
*** gyee has quit IRC05:55
oasizJust an update, but I got the devstack fully installed after the pip downgrade. Since IRC is the best documentation platform, what I did to get this to run on 18.04: You can still find a readily available installer as python3-pip_18.1-5_all.deb  - extracting with ar -x , the structure is /usr/lib/python3 instead of /usr/lib/python3.6 that's with normal apt version -- manually replacing the files and as an extra step I commented the call in i06:01
oasizThere are more elegant solution available but at least this did the trick and script.sh ran well. I assume there will be more questions about this now that stack.sh won't work out of the box with the current guide.06:02
*** Yarboa has quit IRC06:03
*** Yarboa has joined #openstack06:03
*** rcernin has quit IRC06:12
*** tips has quit IRC06:18
*** matt_kosut has joined #openstack06:19
*** hamalq has quit IRC06:21
*** hamalq has joined #openstack06:22
*** pppingme has quit IRC06:32
*** klamath_atx has quit IRC06:47
*** abdysn has joined #openstack06:52
*** gmoro has joined #openstack06:55
*** ladrua has quit IRC06:55
*** ladrua has joined #openstack06:55
*** ladrua has quit IRC06:55
*** ladrua has joined #openstack06:56
*** ladrua has quit IRC06:56
*** ladrua has joined #openstack06:56
*** ladrua has quit IRC06:57
*** ladrua has joined #openstack06:57
*** ladrua has quit IRC06:58
*** ladrua has joined #openstack06:58
*** ladrua has quit IRC06:59
*** ladrua has joined #openstack06:59
*** ladrua has quit IRC06:59
*** ladrua has joined #openstack07:00
*** ladrua has quit IRC07:00
*** ladrua has joined #openstack07:01
*** ladrua has quit IRC07:01
*** ladrua has joined #openstack07:01
*** ladrua has quit IRC07:02
*** ladrua has joined #openstack07:02
*** ladrua has quit IRC07:03
*** ladrua has joined #openstack07:03
*** ladrua has quit IRC07:03
*** ueha has joined #openstack07:18
*** belmoreira has joined #openstack07:21
*** pcaruana has joined #openstack07:46
*** def_jam is now known as eblip07:51
*** slaweq has joined #openstack07:52
*** luksky has joined #openstack07:56
*** pppingme has joined #openstack07:56
*** tesseract has joined #openstack07:56
*** rpittau|afk is now known as rpittau07:59
*** jcapitao has joined #openstack08:00
*** andrewbonney has joined #openstack08:03
*** ChosSimbaOne has left #openstack08:10
*** takamatsu has quit IRC08:13
*** takamatsu has joined #openstack08:15
*** bengates has joined #openstack08:18
*** avisiedo has quit IRC08:46
*** mulbc has quit IRC08:51
*** mulbc has joined #openstack08:53
*** takamatsu has quit IRC08:53
*** takamatsu has joined #openstack08:54
*** avisiedo has joined #openstack08:59
*** Yarboa has quit IRC09:03
*** shyamb has joined #openstack09:07
*** adeberg has quit IRC09:09
*** adeberg has joined #openstack09:12
*** rockie has quit IRC09:15
*** Yarboa has joined #openstack09:15
*** rockie has joined #openstack09:16
*** PabloMartinez has joined #openstack09:48
*** imega has joined #openstack10:06
*** sergiuw has joined #openstack10:12
*** jv has quit IRC10:17
*** jonaspaulo has joined #openstack10:24
*** bengates has quit IRC10:40
*** shyamb has quit IRC10:41
*** bengates has joined #openstack10:41
*** bengates_ has joined #openstack10:42
*** bengates has quit IRC10:46
*** bengates_ has quit IRC10:57
*** bengates has joined #openstack10:57
*** shyamb has joined #openstack10:57
*** waverider has joined #openstack11:09
*** RickDeckard has joined #openstack11:09
*** Dantalion has quit IRC11:10
*** bengates has quit IRC11:13
*** bengates has joined #openstack11:14
*** jonaspaulo has quit IRC11:29
*** Dantalion has joined #openstack11:32
*** nksegos has joined #openstack11:35
*** adrianc has quit IRC11:39
*** jcapitao is now known as jcapitao_lunch11:39
*** bengates_ has joined #openstack11:41
*** bengates_ has quit IRC11:42
*** bengates_ has joined #openstack11:42
*** adrianc has joined #openstack11:44
*** bengates has quit IRC11:45
*** TeamDevOps has joined #openstack11:45
*** adrianc has quit IRC11:46
*** bengates_ has quit IRC11:50
*** TeamDevOps has quit IRC11:50
*** bengates has joined #openstack11:51
*** shyam89 has joined #openstack11:54
*** adrianc has joined #openstack11:55
*** shyam89 has quit IRC11:56
*** shyamb has quit IRC11:58
*** adrianc has quit IRC12:00
*** adrianc has joined #openstack12:04
*** bbowen has quit IRC12:05
*** bbowen has joined #openstack12:08
*** mrtadis has quit IRC12:15
*** manuvakery_ has quit IRC12:22
*** Diplomat has joined #openstack12:33
DiplomatHey! Can anyone help me with networking please. I created a VM with a private network (10.100.0.0/16) and 2 VMs can connect to each other without any issues + both can ping the gateway IP 10.100.0.1, but unfortunately they don't have external internet connection. I created a public network with a public subnet and set up router and assigned floating IPs to both VMs but yeah. No sure what to do.12:38
DiplomatWhen I'm pinging 8.8.8.8 for example then I can see packets in tap, bridge and vxlan, but those packets don't want to go to the public interface for some reason.12:39
DiplomatThis is my linuxbridge config in controller: https://dpaste.org/Gtec and this is in my controller https://dpaste.org/fKgN12:41
Diplomatcompute*12:43
DiplomatAlso, floating IP addresses are pingable12:44
*** jcapitao_lunch is now known as jcapitao12:52
*** bengates has quit IRC13:06
*** damien_r has quit IRC13:06
*** bengates has joined #openstack13:08
*** abdysn has quit IRC13:12
*** Dantalion has quit IRC13:18
*** damien_r has joined #openstack13:19
*** jv has joined #openstack13:20
*** Dantalion has joined #openstack13:21
admin0Diplomat, how does it look inside the router namespace ?13:23
Diplomatadmin0: It was the same in the router namespace13:24
admin0you need to login inside the router namespace and then check if the ips are correect, gw is correct13:24
admin0and then from the namespace find out the bridge13:25
admin0and from the bridge to the outgoing interface13:25
*** ueha has quit IRC13:29
Diplomatadmin0: When I do route -n in router namespace then the gateway is correct, yes13:31
Diplomat10.100.0.0 -> 0.0.0.0 -> my public subnet gateway13:32
DiplomatI'm not super sure how to find the bridge and outgoing interface from there13:32
*** slaweq has quit IRC13:34
DiplomatAh interesting.. The gateway interface is qg-6a3e54ca-63 but when I do ifconfig it shows a wrong IP address there. The gateway is A.B.C.97 but qg-6a3e54ca-63's IP is A.B.C.10013:34
DiplomatYeah, A.B.C.100 is set as "External Gateway" in the router13:36
*** slaweq has joined #openstack13:36
*** sshnaidm has quit IRC13:38
*** cmorey has joined #openstack13:41
*** Goneri has joined #openstack13:43
*** sshnaidm has joined #openstack13:46
*** sshnaidm has quit IRC13:48
*** damien_r has quit IRC13:49
*** rockie has quit IRC13:57
*** zbsarashki has joined #openstack13:59
*** pcaruana has quit IRC14:03
*** admin0 has quit IRC14:03
*** chrome0 has quit IRC14:04
*** chrome0 has joined #openstack14:06
*** pcaruana has joined #openstack14:14
*** damien_r has joined #openstack14:27
*** morazi has quit IRC14:29
*** tips has joined #openstack14:32
*** sergiuw has quit IRC14:40
*** morazi has joined #openstack14:42
*** admin0 has joined #openstack14:48
*** sergiuw has joined #openstack14:49
*** pcaruana has quit IRC15:01
*** pcaruana has joined #openstack15:02
*** nurdie has joined #openstack15:15
*** waverider has quit IRC15:16
LarsErikPHas anyone tried to nova with vgpu on the A100 yet? I'm experiencing some weird bugs..15:17
*** mrtadis has joined #openstack15:33
*** RickDeckard has quit IRC15:35
*** pcaruana has quit IRC15:36
*** arnoldoree has quit IRC15:43
*** pcaruana has joined #openstack15:49
*** jmcvaughn has quit IRC15:49
*** jmcvaughn has joined #openstack15:51
*** irclogbot_0 has quit IRC15:53
jrosserLarsErikP: what are you seeing?15:56
*** irclogbot_3 has joined #openstack15:58
*** sshnaidm has joined #openstack16:05
*** irclogbot_3 has quit IRC16:11
*** irclogbot_0 has joined #openstack16:13
*** RickDeckard has joined #openstack16:13
*** macz_ has joined #openstack16:15
*** macz_ has quit IRC16:17
*** noogie has quit IRC16:26
*** bengates has quit IRC16:40
*** bengates has joined #openstack16:42
*** noogie has joined #openstack16:49
*** bengates_ has joined #openstack16:50
*** bengates has quit IRC16:52
*** jcapitao has quit IRC16:57
*** bengates_ has quit IRC16:59
*** bengates has joined #openstack17:00
*** rpittau is now known as rpittau|afk17:01
*** ladrua has joined #openstack17:06
*** nksegos has quit IRC17:09
*** gyee has joined #openstack17:23
*** tesseract has quit IRC17:30
*** samuelb has quit IRC17:40
*** samuelbernardo has joined #openstack17:41
*** bengates has quit IRC17:43
*** jangutter_ has joined #openstack18:00
*** jangutter has quit IRC18:02
*** jangutter_ has quit IRC18:21
*** jangutter has joined #openstack18:22
*** jangutter has quit IRC18:39
*** jangutter has joined #openstack18:39
*** andrewbonney has quit IRC18:40
*** jangutter has quit IRC18:40
*** jangutter has joined #openstack18:41
*** pppingme has quit IRC18:49
DiplomatAlrighty, I got my gateway IP correct, but yeah.. it didn't help. It seems like bridge is not sending traffic to the public Nic for some reason18:55
DiplomatI can't find any errors too to see what's wrong18:55
DiplomatHere's tcpdump from my bridge in compute node https://dpaste.org/XELC18:58
DiplomatHere's route -n from my router https://dpaste.org/vFMC19:04
DiplomatAlso, it's possible to ping the private subnet's gateway and public subnet's gateway from the router: https://dpaste.org/T1HJ19:08
LarsErikPjrosser: it's using SR-IOV, and thus exposes 16 VFs as resource providers to placement. When a physical card is filled up with vGPUs, nova-compute from some reason tries to update a (seemingly random) resource provider, and throws a python stacktrace.. I can post the stacktrace tomorrow19:08
jrosserLarsErikP: I’ve done vgpu prototype on T4 (I don’t have my A100 yet) and done it with vgpu/mdev19:10
LarsErikPjrosser: is the T4 using for SR-IOV as well?19:10
jrosseryou can only divide into a maximum of 7 parts afaik and it’s very different to sriov on a nic19:11
jrosseryou’ve got the nvidia grid driver?19:11
LarsErikPyeah. vgpu 11.219:11
LarsErikPit ships a tool to enable sr-iov on the card, which is needed to utilize vGPU (not running MIG), and that tool creates 16 VFs..19:12
jrosserwith the t4 out of the box and no enabling of sriov I was able to follow this https://docs.openstack.org/nova/latest/admin/virtual-gpu.html and get it working relatively easily, like 1 day of fiddling19:14
jrossermostly spent forcing the grid driver to install on Ubuntu when it demands RHEL19:15
*** sshnaidm has quit IRC19:15
LarsErikPright. I've also had success with the V100 - but that card don't need sr-iov19:15
LarsErikPwow. you actually found a stable way to run that driver on ubuntu?19:16
LarsErikPI just didn't bother, and we ended up with running CentOS on the GPU nodes19:16
jrosserunpack the driver .run file and symlink up some lib64 dirs to be in the same place as centos/rhel and it seems ok19:17
LarsErikPinteresting..19:17
jrosserthere’s a -x or something which unpacks it rather than installs19:17
*** sshnaidm has joined #openstack19:18
jrossertbh I think with the t4 I would have been using mig though19:18
*** ladrua has quit IRC19:19
LarsErikPbut does the T4 support sr-iov?19:19
*** ladrua has joined #openstack19:19
*** takamatsu has quit IRC19:19
jrosserwell I’m not sure, and as I say I don’t yet have an a100 but was expecting to do mig on them both19:21
*** nksegos has joined #openstack19:35
LarsErikPi don't think MIG will make a difference here..19:35
LarsErikPyou would still need nova to create a mdev for the vGPU19:36
*** nksegos has quit IRC19:36
LarsErikPbut you managed to use the T4 with vgpu, without enabling sr-iov? That's interesing. I could try on the A100 as well, but I don't think it will work at all. Because without enablig sriov, there is no mdev_bus in sysfs..19:37
*** luksky has quit IRC19:43
*** cmorey has quit IRC19:43
*** luksky has joined #openstack19:43
jrosserLarsErikP: I can take a closer look tomorrow19:46
LarsErikPthanks!19:48
*** klamath_atx has joined #openstack19:54
*** nksegos has joined #openstack20:07
*** Zoup has joined #openstack20:12
*** pcaruana has quit IRC20:27
*** imega has quit IRC20:28
*** damien_r has quit IRC20:47
*** belmoreira has quit IRC20:53
*** ladrua has quit IRC21:20
*** rcernin has joined #openstack21:24
*** matt_kosut has quit IRC21:36
*** RickDeckard has quit IRC21:37
*** jmcvaughn has quit IRC21:42
*** slaweq has quit IRC21:47
*** matt_kosut has joined #openstack21:48
*** slaweq has joined #openstack21:49
*** matt_kosut has quit IRC21:49
*** jmcvaughn has joined #openstack21:50
*** matt_kosut has joined #openstack21:53
*** matt_kosut has quit IRC21:57
*** gyee has quit IRC22:04
*** gyee has joined #openstack22:05
*** Goneri has quit IRC22:07
*** ddstreet has quit IRC22:10
*** ddstreet has joined #openstack22:10
*** Goneri has joined #openstack22:27
*** muhaha has joined #openstack22:39
*** luksky has quit IRC22:42
*** luksky has joined #openstack22:56
*** slaweq has quit IRC22:56
*** nurdie has quit IRC23:05
*** Zoup has quit IRC23:10
*** sergiuw has quit IRC23:17
*** nurdie has joined #openstack23:21
*** nurdie has quit IRC23:25
*** luksky has quit IRC23:26
*** aluria has quit IRC23:33
*** aluria has joined #openstack23:34
*** wallacer has quit IRC23:34
*** muhaha has quit IRC23:37
*** klamath_atx has quit IRC23:39
*** wallacer has joined #openstack23:40
*** klamath_atx has joined #openstack23:47
*** sergiuw has joined #openstack23:52

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!