Tuesday, 2019-01-15

*** aaronsheffield has quit IRC00:09
*** sofioni has quit IRC00:28
*** AniruddhaSG has quit IRC01:09
*** sthussey has quit IRC01:22
openstackgerritMerged openstack/airship-treasuremap master: Align Airship Seaworthy pipeline wiht latest ceph  https://review.openstack.org/63079302:47
*** roman_g has quit IRC05:16
*** Bhujay has joined #airshipit05:43
*** Bhujay has quit IRC05:44
*** Bhujay has joined #airshipit05:45
*** Bhujay has quit IRC05:46
*** Bhujay has joined #airshipit05:46
*** v1k0d3n has joined #airshipit05:52
*** Bhujay has quit IRC07:01
*** roman_g has joined #airshipit07:04
*** Bhujay has joined #airshipit07:10
*** Bhujay has quit IRC07:11
*** Bhujay has joined #airshipit07:11
*** dimitris_ has joined #airshipit08:12
openstackgerritFilippo Inzaghi proposed openstack/airship-promenade master: Add airship-discuss  https://review.openstack.org/62299808:27
openstackgerritFilippo Inzaghi proposed openstack/airship-divingbell master: Add airship-discuss  https://review.openstack.org/62299708:27
*** nick_kar has joined #airshipit10:14
*** nick_kar has quit IRC10:28
*** nick_kar has joined #airshipit10:29
*** roman_g has quit IRC11:12
*** openstackgerrit has quit IRC11:22
*** pkaralis has joined #airshipit12:10
*** AniruddhaSG has joined #airshipit12:53
*** sthussey has joined #airshipit13:32
*** Kaspars has joined #airshipit13:50
mattmceuen#startmeeting airship14:00
openstackMeeting started Tue Jan 15 14:00:20 2019 UTC and is due to finish in 60 minutes.  The chair is mattmceuen. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: airship)"14:00
openstackThe meeting name has been set to 'airship'14:00
mattmceuen#topic rollcall14:00
*** openstack changes topic to "rollcall (Meeting topic: airship)"14:00
mattmceuengm everyone!14:00
dwalt\o/14:00
dwaltGM!'14:00
mattmceuenHere's our agenda for today: https://etherpad.openstack.org/p/airship-meeting-2019-01-1514:00
mattmceuenPlease add anything you'd like to discuss14:01
evgenylGood morning!14:01
mattmceuenand we'll give it a couple mins for folks to filter in14:01
sthusseyhere14:01
Kasparsalso here14:01
mattmceuenhey guys14:01
portdirecto/14:02
*** aaronsheffield has joined #airshipit14:03
mattmceuenso the meat of the agenda today is to share/explain a few things that have gotten added in recently14:03
mattmceuen#topic podsecuritypolicy chart14:03
*** openstack changes topic to "podsecuritypolicy chart (Meeting topic: airship)"14:03
*** powerds0111 has joined #airshipit14:03
mattmceuenSo you may or may not be familiar with k8s PodSecurityPolicies14:03
mattmceuenIf you configure your k8s to use them, then the k8s api server will only allow you an actor to schedule pods that meet certain criteria/policy14:04
mattmceuenBased on k8s RBAC14:04
mattmceuenThis is a security feature we wanted to add into airship, but in a way that doesn't break all the privileged actions that are taken across airship and openstack-helm14:05
mattmceuenThis helm chart was added to openstack-helm-infra: https://github.com/openstack/openstack-helm-infra/tree/master/podsecuritypolicy14:05
mattmceuenIt specifies by default an incredibly (100%) permissive podsecuritypolity, and sets it up as a default for the cluster14:06
mattmceuenIt was also added into the Treasuremap reference yamls: https://review.openstack.org/#/c/629686/14:06
mattmceuenYou can do a couple things through the chart:14:07
sthusseyWhy was the chart put in a different project if it was intended as a part of Airship?14:07
mattmceuenbecause it's not airship-specific14:07
mattmceuenpretty broadly applicable14:07
mattmceuenYou can use the chart to:14:08
mattmceuen1) add whatever additional podsecuritypolicies you want for your cluster, programmatically, letting helm manage lifecycle14:08
mattmceuen2) change the default(s)14:08
mattmceuenOver time, we want to tune the defaults in the chart to be a reasonable non-fully-open set of policy, as much as possible.  However, the intent is also that operators fully customize it for their workloads,14:09
portdirectNice work mattmceuen14:09
mattmceuenYou can set up defaults in the chart individually for serviceacounts, authenticated users, and unauthenticated users via the chart, and/or associate the PSPs to other roles outside of the chart itself14:09
*** cbcg has joined #airshipit14:10
mattmceuenthat's it - any questions?14:10
*** roman_g has joined #airshipit14:10
portdirectShould help make it much easier to manage the security posture of a cluster.14:10
roman_go/14:10
hogepodgeHi14:10
mattmceueno/ guys :)14:10
mattmceuenOne more note, just having to do with how PSP's work in k8s14:10
sthusseyWhat is the expectation of coordinating access to PSPs and the service accounts created by other Helm charts?14:10
mattmceuenthat'll be on the operator to coordinate; in an airship world, via substitution.  That would be a good pattern to construct/demo in treasuremap I'd think14:11
mattmceuenso PSPs only affect pods created *after* the PSPs are in place.  So, that's why the psp chart was added first in the treasuremap armada manifest14:12
mattmceuenalright, next topic:14:13
mattmceuen#topic Mini-mirror integration14:13
*** openstack changes topic to "Mini-mirror integration (Meeting topic: airship)"14:13
mattmceuendwalt take it away!14:13
dwaltalrighty!14:14
roman_gdwalt: Nice job! I like it. Thank you.14:14
dwaltOne of the challenges we were having in larger scale deployments was the coordination of packages on the host14:14
dwaltThanks roman_g!14:14
dwalti.e. packages changed between the time that we ran test deployment pipelines to the time we actually deployed Airship14:15
dwaltMini-mirror exists as a way to combat that issue. By mirroring Ubuntu package repositories and deploying them into the cluster, we can control what packages exist on the host for a deployment14:17
powerds0111o/14:17
dwaltCurrently, you can utilize mini-mirror by building an image by specifying the sources/packages you would like to mirror14:18
dwalt#link https://github.com/openstack/openstack-helm-images/blob/master/mini-mirror/README.rst14:18
dwaltThe next step is integrating mini-mirror into the genesis process, with the vision that we may one be able to deploy Airship without internet connectivity14:19
dwaltThe chart for mini-mirror is also here:14:19
dwalt#link https://github.com/openstack/openstack-helm-addons/tree/master/mini-mirror14:19
dwaltAny questions?14:19
Kasparsthis looks really cool, are you thinking to add it to treasuremap any time soon?14:19
roman_gAt the moment mini-mirror does not contain: docker image archives and k8s binaries; they would need to be baked into some /var/cache inside the image14:20
dwaltKaspars: Thanks! Definitely. As soon as we have a published demo image, we would like to get a full demo14:20
*** srwilkers has joined #airshipit14:21
sthusseymini-mirror is just a framework. It should contain whatever packages you tell it sourced from whatever source you provide when you build the image.14:21
dwalt++14:22
mattmceuenYup roman_g those would be good enhancements down the road14:22
mattmceueno/ powerds011114:22
mattmceuenanything else on the mini-mirror front guys?14:22
mattmceuenalso, hooray for clever names.  very catchy14:22
*** Soopaman has joined #airshipit14:23
sthusseyI would suggest if it isn't merged yet that you consider naming it mini-apt or something14:23
mattmceuenlol well you got one vote each way there14:23
mattmceuenbut it's merged14:23
sthusseyIt isn't really part of Airship, so this is just tangential, but there is a merged spec in Airship to support multiple OSes and making mini-mirror Ubuntu specific doesn't really jive with that14:24
portdirectThere would be nothing stopping adding rpm support14:24
mattmceuentotally agree14:24
sthusseyAnd I don't think long term adding the complexity to make a single framework support both is worth it14:24
sthusseyJust pick whether you want mini-apt or mini-rpm and go forward14:24
portdirectOr just a value toggel14:25
portdirectAll it would take is turning on or off, two manifests14:25
mattmceuenthe *chart* part is really an nginx chart, right dwalt?14:25
sthusseyYeah, I don't really care about the chart part14:25
mattmceuenand the apt magic is in the docker build stuff14:25
sthusseyThe complexity is in building the image14:25
dwaltAt it's core, it really is just NGINX14:25
dwaltthe image itself is just an NGINX image with the mirrored snapshots14:25
sthusseyAgain, just tangential as it isn't part of Airship. But a consideration.14:26
dwaltso a similar tool could be used (with a toggle) to mirror rpm repos14:26
mattmceuenit might make sense to separate the apt docker build from the rpm docker build, since they'd likely be quite different14:26
dwaltThis is true. And OpenStack-Helm addons already has the pattern for multiple OS/Dockerfiles14:26
mattmceuenI don't think it's tangential, we want to bring multi-os support into Airship14:27
dwaltOpenStack-Helm-images*14:27
mattmceuengood point14:27
mattmceuenanything else on MM?14:27
portdirectMaxi-mirror?14:27
srwilkers:)14:28
mattmceuenJust depends how much stuff you shove in the image portdirect :)14:28
mattmceuen#topic Monthly Treasuremap Release14:28
portdirectIs there thought toward caching of things like docker images at this point, or that still at design stage?14:28
*** openstack changes topic to "Monthly Treasuremap Release (Meeting topic: airship)"14:28
* mattmceuen jumps the gun14:28
mattmceuen#topic maxi-mirror14:28
*** openstack changes topic to "maxi-mirror (Meeting topic: airship)"14:28
mattmceuenportdirect: I think there's been some discussion around it, but I don't know if any stories are in storyboard or anyone working on it yet14:29
roman_gme neither14:29
sthusseyWhile possibly branded as mini-mirror, I would expect something like that to basically be completely separate in terms of image build14:29
portdirectI was more thinking an adjunt here, to separate the concerns14:30
sthusseyI think designs right now around image caching are something like squid w/ MITM TLS signing14:30
portdirectLooks like mini-mirror allows offline host provisioning14:30
roman_gdwalt: would you work on blueprint, or I can take it?14:31
portdirectBut still leaves the gap of how to get containers into a site without internet or artifactory or summat.14:31
roman_g(bake into genesis iso)14:31
dwaltroman_g: You can take a stab at it if you'd like!14:31
roman_gdwalt: deal14:31
sthusseyAt this point, that wasn't the business value targeted w/ mini-mirror14:31
portdirectNice14:31
portdirectsthussey: I understand that, hence my q.14:32
roman_gsthussey: portdirect: mattmceuen what is your decision/input?14:32
sthusseyDecision on what? If you want to draw up a spec for something, go for it.14:33
mattmceuenroman_g:  if you'd like to take a stab at a spec PS, I think that gives a good opportunity to prompt discussion, even (especially) if that prompts a change in direction14:33
portdirectAfter 3 mins of percolation time, bit early for me to put my stake in the ground on anything roman_g14:33
portdirect++ to mattmceuen s point14:34
mattmceuenthanks roman_g14:34
mattmceuenAnything else on the multi-mirror discussino?14:34
mattmceuen#topic Monthly Treasuremap Release14:35
*** openstack changes topic to "Monthly Treasuremap Release (Meeting topic: airship)"14:35
mattmceuenKaspars, all yours14:35
*** ab2434_ has joined #airshipit14:35
Kasparsgreat! so we have I would say almost routine monthly tag of airship-treasuremap manifests found located at https://github.com/openstack/airship-treasuremap/releases14:36
Kasparsas before - it includes auto-uplifts of components (with a few exclusions this time that include MAAS HA that Scott will talk about later, and armada due to test pod handling)14:36
Kasparsthere are a few other notable changes as ceph partition handling and update of Docker to much newer version14:37
Kasparsas before - it has ran through full airship deployment pipeline (Airship Seaworthy), here are few sentances about it https://airship-treasuremap.readthedocs.io/en/latest/14:38
KasparsI would say enjoy the latest release candidate for 1.014:38
mattmceuenthat's awesome Kaspars14:39
seaneaganin-flight fix for Armada test pod handling: https://review.openstack.org/#/c/629313/14:39
mattmceuenthanks for keeping the good stuff coming and the pipelines green :)14:39
mattmceueno/ seaneagan14:39
mattmceuenany other discussion on the release candidate?14:40
mattmceuen#topic HA MaaS overview14:41
*** openstack changes topic to "HA MaaS overview (Meeting topic: airship)"14:41
mattmceuensthussey all yours!14:41
sthusseyRecently there a significant refactor of the Airship chart for the Canonical MAAS product.14:42
sthusseyPreviously it was a bit fragile in terms of where it ran and how it responded to moving to other nodes.14:42
sthusseyIt now should support a highly available configuration for the MAAS components (HA Postgres under it is out of scope) and with that HA support comes mobility across nodes in the cluster14:43
sthusseyThere are some caveats and suppositions for the current solution14:44
*** sofioni has joined #airshipit14:44
portdirectthats great sthussey14:44
portdirectis the required config in treasuremap and airship in a bottle - so we can take it for a spin?14:44
sthussey1) It expects a flat L2 broadcast domain for the PXE network across all nodes in the cluster14:44
sthusseyre: config - Can't speak to Treasuremap, but the multinode-gate in AIAB should work with it14:45
Kasparsthere has not been patchsets submitted for treasuremap, yet14:46
sthussey2) You must be able to route between the PXE booting network and whatever network your cluster ingress runs on. If there is a single network in your cluster for all traffic, that is sufficient.14:46
sthussey3) Currently there is no TLS support for accessing the MAAS API/metadata server. That support will hopefully be in place in the near future14:47
hogepodgeSomewhat related, is Ironic making progress in the RCs?14:47
mattmceuenhogepodge:  there was good discussion in rodolfo's zoom call last thursday, if you want to see the recording14:48
sthussey4) If you are using the OSH-infra ingress chart, you must be using nginx-ingress-controller v0.20.0 or better14:48
hogepodge(kind of previous topic juggling multiple chat windows)14:48
sthusseyOtherwise you get false port binding conflicts between cluster ingress and maas ingress14:48
sthusseyThat is all from me.14:48
*** Soopaman has quit IRC14:48
mattmceuenthanks for all the details sthussey - that was a big effort with a lot of tricky bits14:49
hogepodgemattmceuen yes please, I’ll coordinate offline thx14:49
mattmceuenawesome, sounds good hogepodge14:49
mattmceuensounds like treasuremap is pinned to an old version - I guess the interface to the maas chart isn't backwards compatible, right sthussey?14:49
sthusseyNot sure, would have to test. The previous overrides certainly won't enable the HA functionality14:50
Kasparsthe pipeline did fail - so I had to disable auto-uplift for maas/drydock charts..14:51
mattmceuengotcha.  Kaspars, maybe we can coordinate in the chat here to after the meeting to see what the issues you were seeing are, so we can get HA MaaS into TM now that the stable release was cut14:51
mattmceuenk we have a few more topics and scant few minutes!14:52
mattmceuen#topic PS needing review14:52
*** openstack changes topic to "PS needing review (Meeting topic: airship)"14:52
Kasparsthere was a pretty large PS comitted to AIAB - so those changes need to be ported to treasuremap as well14:52
mattmceuenI'm just going to paste these with a blurb and then move on!14:52
Kasparsto support latest MAAS/drydock - as well as few DNS handling need to be settled14:52
Kasparssome work..14:52
mattmceuenBlurb:  I want to make sure there's an opportunity for folks to bring up PS weekly that they view as high priority, and/or lost in the shuffle14:52
mattmceuen    https://review.openstack.org/63074914:52
mattmceuen    https://review.openstack.org/63075214:52
mattmceuen    https://review.openstack.org/63075514:52
mattmceuen^^ from dwalt - please give those some eyes soon team14:53
mattmceuenany others?14:53
mattmceuenOne topic not on the agenda:14:54
mattmceuen#topic best of luck mark-burnett !14:54
*** openstack changes topic to "best of luck mark-burnett ! (Meeting topic: airship)"14:54
hogepodgeI have one regarding ATC codes for Denver Summit14:54
Kasparsbest of luck, Mark!!14:55
mattmceuenMark is taking another opportunity.  He is of course more than welcome to continue participating in the airship community as much as possible!  But I also just wanted to share that hey may not have the same bandwidth14:55
mattmceuenMark has been instrumental in forming the airship community, and definitely with him best of luck in the next endeavor!14:56
mattmceuen#topic ATC codes for Denver Summit14:56
*** openstack changes topic to "ATC codes for Denver Summit (Meeting topic: airship)"14:56
mattmceuengo for it hogepodge14:56
hogepodge Congratulations Mark!14:57
cbcgcongarts Mark!14:57
hogepodgewe need to put together a list of active contributors to receieve discount codes for the Denver summit14:57
hogepodgeTo help us with that we need a list of repositories to draw from for contributions.14:58
*** georgk has joined #airshipit14:58
hogepodgeI’m assuming all repos prefixed airship-, but if there are others we missed getting them to fungi would be a big help14:58
mattmceuenI think that is the right set hogepodge14:59
mattmceuenthanks for making sure14:59
hogepodgeSince Airship is so new, we’re looking at contributions back to the launch of Airship as a pilot project in Vancouver for the contribution timeframe14:59
hogepodgecool, thanks :-)14:59
mattmceuenSounds reasonable to me! :)14:59
mattmceuenOk guys we are hard outta time -- please vote in the board election, and help out with the deckhand doc builds:15:00
mattmceuen    Please, assist: airship-deckhand documentation builds are broken since Oct. 30, 201815:00
mattmceuen    Bug: https://storyboard.openstack.org/#!/story/200474715:00
mattmceuen    Logs: https://readthedocs.org/api/v2/build/8395458.txt15:00
roman_gIt's mine, and I don't know how to fix it.15:01
dwaltroman_g: I will take a look at that sometime today15:01
mattmceuenCool - let's continue the discussion here post-meeting roman_g and get that working15:01
mattmceuen#endmeeting15:01
*** openstack changes topic to "airshipit.org || General Review Dashboard: https://review.openstack.org/#/q/project:%255Eopenstack/airship.*+status:open,n,z"15:01
mattmceuenthanks all!15:01
openstackMeeting ended Tue Jan 15 15:01:38 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/airship/2019/airship.2019-01-15-14.00.html15:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/airship/2019/airship.2019-01-15-14.00.txt15:01
roman_gThank you.15:01
openstackLog:            http://eavesdrop.openstack.org/meetings/airship/2019/airship.2019-01-15-14.00.log.html15:01
dwaltI have been working on some of the gates15:01
mattmceuennote some folks need to go to the OSH meeting and so may be a little laggy :)15:01
*** Bhujay has quit IRC15:02
*** jezogwza has joined #airshipit15:04
*** Kaspars has quit IRC15:05
*** ab2434_ has quit IRC15:12
*** openstackgerrit has joined #airshipit15:43
openstackgerritMerged openstack/airship-treasuremap master: Update docs, execute deploy_site.sh when in shipyard directory  https://review.openstack.org/62906315:43
hogepodgemattmceuen: rodolfo: I have a docker-compose and loci Ironic implementation (both standalone and as nova driver) if you want to see how I handled things like tftp and image management15:47
*** dimitris_ has quit IRC15:49
openstackgerritMerged openstack/airship-divingbell master: Add airship-discuss  https://review.openstack.org/62299716:22
*** georgk has quit IRC16:49
*** pkaralis has quit IRC16:50
*** nick_kar has quit IRC16:51
*** roman_g has quit IRC17:07
mark-burnettThanks Matt & all -- sorry I missed the meeting this morning18:09
mark-burnett:)18:09
openstackgerritLev Morgan proposed openstack/airship-pegleg master: pki: Port Promenade's PKI catalog into Pegleg  https://review.openstack.org/60613118:11
openstackgerritLev Morgan proposed openstack/airship-pegleg master: pki: Port Promenade's PKI catalog into Pegleg  https://review.openstack.org/60613118:16
mattmceuenno worries mark-burnett :)18:34
mattmceuenhogepodge - would love to see that!18:34
hogepodgemattmceuen: it's all very rough, but the one based on bifrost is here: https://github.com/hogepodge/symmetrical-memory18:35
hogepodgethe one that is a driver for nova is here https://github.com/hogepodge/container-ironic18:35
mattmceuenthanks for sharing those, will take a look and pass them to folks looking at ironic18:36
openstackgerritMark Burnett proposed openstack/airship-promenade master: Expand bootstrapping documentation  https://review.openstack.org/63105018:56
openstackgerritMark Burnett proposed openstack/airship-promenade master: Expand bootstrapping documentation  https://review.openstack.org/63105019:04
openstackgerritLev Morgan proposed openstack/airship-pegleg master: pki: Port Promenade's PKI catalog into Pegleg  https://review.openstack.org/60613119:29
*** AniruddhaSG has quit IRC19:29
openstackgerritNishant Kumar proposed openstack/airship-specs master: (divingbell) Ansible framework  https://review.openstack.org/62822119:30
*** shoaibwr has quit IRC19:31
*** shoaibwr has joined #airshipit19:31
openstackgerritNishant Kumar proposed openstack/airship-specs master: (divingbell) Ansible framework  https://review.openstack.org/62822119:32
*** mfuller has joined #airshipit19:45
openstackgerritShoaib Nasir proposed openstack/airship-armada master: Add support in Armada CLI to pass user bearer tokens to tiller  https://review.openstack.org/63075419:50
mfullerHello, I'm a dev at SUSE and I'm trying to get a dev environment set up following the treasuremap documentation. I was curious if it's possible to utilize some of Promenade's functionality for k8s node lifecycle management (node labeling, etc.) in a BYO k8s environment, without bootstrapping19:55
sthusseyshould be19:55
mfullerFor example, is it possible to set up site definition documents to describe an already existing environment, and then use update_site to perform labeling changes?19:56
openstackgerritShoaib Nasir proposed openstack/airship-armada master: Add support in Armada CLI to pass user bearer tokens to tiller  https://review.openstack.org/63075419:59
openstackgerritSean Eagan proposed openstack/airship-armada master: Add configurability of uninstall timeout  https://review.openstack.org/63077920:02
*** cbcg has quit IRC20:52
*** georgk has joined #airshipit20:59
georgkmattmceuen: ping!21:06
sthussey@mfuller correct21:14
mfullerI'm still working my way though all the documentation so I may not have gotten there yet, but is there a section or an example of the minimum level of detail Shipyard needs to know about a site in order to perform updates this way?21:24
mfullerWould it need to know low-level hardware details about nodes?21:26
openstackgerritDrew Walters proposed openstack/airship-deckhand master: CI: Fix integration job  https://review.openstack.org/63074921:29
*** roman_g has joined #airshipit21:36
openstackgerritLev Morgan proposed openstack/airship-pegleg master: CLI capability to generate and encrypt passphrases  https://review.openstack.org/60542521:50
openstackgerritDrew Walters proposed openstack/airship-treasuremap master: airskiff: Use Docker CE 18.06 for Minikube  https://review.openstack.org/62354922:00
openstackgerritDrew Walters proposed openstack/airship-treasuremap master: airskiff: Use Minikube for Kubernetes deployment  https://review.openstack.org/62261022:00
openstackgerritDrew Walters proposed openstack/airship-treasuremap master: airskiff: Use Minikube for Kubernetes deployment  https://review.openstack.org/62261022:31
sthussey@mfuller I'd actually need to walk through it myself - I didn't design or implement it.22:38
sthusseyBut it does require drydock in the middle at this point because drydock dynamically renders some labels22:38
openstackgerritDrew Walters proposed openstack/airship-treasuremap master: airskiff: Use Minikube for Kubernetes deployment  https://review.openstack.org/62261023:16
*** georgk has quit IRC23:28
*** georgk has joined #airshipit23:52

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!