| *** jamielennox|away is now known as jamielennox | 00:02 | |
| *** pmannidi has quit IRC | 00:03 | |
| *** pmannidi has joined #openstack-dib | 00:25 | |
| *** pmannidi is now known as pmannidi|Trg | 00:25 | |
| *** pmannidi|Trg is now known as pmannidi | 00:25 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Switch debian to deb.debian.org https://review.openstack.org/466182 | 01:07 |
|---|---|---|
| openstackgerrit | Merged openstack/diskimage-builder master: Add weights to digraph https://review.openstack.org/466473 | 01:07 |
| *** Sukhdev has joined #openstack-dib | 01:47 | |
| *** jamielennox is now known as jamielennox|away | 02:00 | |
| *** adam_g has quit IRC | 02:24 | |
| *** adam_g has joined #openstack-dib | 02:25 | |
| *** adam_g has quit IRC | 02:32 | |
| *** adam_g has joined #openstack-dib | 02:35 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Move exception to it's own file (again) https://review.openstack.org/466129 | 02:54 |
| *** adam_g has quit IRC | 03:24 | |
| *** adam_g has joined #openstack-dib | 03:24 | |
| *** jamielennox|away is now known as jamielennox | 03:53 | |
| *** xinliang has quit IRC | 03:54 | |
| *** xinliang has joined #openstack-dib | 03:55 | |
| *** xinliang has quit IRC | 03:55 | |
| *** xinliang has joined #openstack-dib | 03:55 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Split partition into it's own file https://review.openstack.org/465837 | 04:29 |
| openstackgerrit | Merged openstack/diskimage-builder master: Move parts of Partition creation into object https://review.openstack.org/465838 | 04:29 |
| *** adam_g has quit IRC | 04:30 | |
| *** adam_g has joined #openstack-dib | 04:32 | |
| *** andreas-f has joined #openstack-dib | 04:35 | |
| *** isaacb has joined #openstack-dib | 04:59 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Unit testing for graph generation https://review.openstack.org/466591 | 05:13 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at networkx for digraph implementation https://review.openstack.org/466592 | 05:13 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: Revert "Set manifests to mode 600 and owner root" https://review.openstack.org/466593 | 05:13 |
| ianw | yolanda / andreas-f : i'm fairly convinced we don't have to maintain our own graph implementation. ^^ very WIP ... but i think it's quite close | 05:14 |
| *** isaacb has quit IRC | 05:41 | |
| andreas-f | ianw: can you please recheck, that the networkx comes with all what we need? IMHO e.g. a topological sort that respects the weights is missing. | 05:47 |
| andreas-f | Also there is no appropriate iterator. | 05:48 |
| ianw | andreas-f: i can't imagine we can't do what we want with networkx | 06:05 |
| ianw | there is no way that sorting our digraph is something so special it's not handled | 06:05 |
| ianw | but, i haven't fully explored it, yet | 06:06 |
| ianw | even if the exact sort isn't there, i'd prefer we use it as the base and simply add a small sorting function | 06:06 |
| andreas-f | ianw: Please check! I'd really love to remove our implementation. But I had a look some month ago for an appropriate graph lib - and did not find anything that fulfills all requirements. | 06:07 |
| andreas-f | It's not a small sorting function: you have to re-implement the topological sort. | 06:07 |
| *** aparnav has joined #openstack-dib | 06:09 | |
| ianw | andreas-f: i'll keep looking, even if something in https://networkx.github.io/documentation/development/reference/algorithms.html doesn't fit exactly, we'd be better spending our time contributing something upstream that does | 06:11 |
| ianw | then everyone can use it | 06:11 |
| ianw | in the mean time, even if we just keep things in a nx.Graph, it still exposes the node & edge information so you can write your own sorting function | 06:12 |
| yolanda | ianw, is that ok to add as a dependency in global requirements? we couldn't add the past graph lib that we explored, because it was unmaintained | 06:16 |
| ianw | yolanda: yep, it's in requirements already | 06:16 |
| yolanda | nice | 06:16 |
| ianw | i think the DAG stuff might fit the bit ... | 06:17 |
| ianw | anyway, as the firs step adding some unit tests for config-file -> graph as done in 455691 will be our first stop | 06:18 |
| andreas-f | A short list of things that must be changed / added when switching to nx: | 06:29 |
| andreas-f | * Concept: Own classes cannot inherit from nx.Node / nx.Edge | 06:29 |
| andreas-f | * add_node with check of unique name | 06:29 |
| andreas-f | * get_iter_node_values | 06:29 |
| andreas-f | * topological_sort | 06:29 |
| andreas-f | * Node.get_iter_outgoing | 06:29 |
| *** isaacb has joined #openstack-dib | 06:33 | |
| *** isaacb has quit IRC | 06:33 | |
| ianw | andreas-f: i think you're misunderstanding nx a bit. you don't inherit from their classes. you just put in an hashable key. and each node/edge can have arbitrary attributes | 06:37 |
| ianw | so each node is hashed based on "name" | 06:37 |
| ianw | and we can have an obj attribute, that is our "plugin" | 06:37 |
| ianw | we can keep a dict of names and simply check if it's already there | 06:38 |
| ianw | i don't see why, when building partitions, the node can't look at its outgoing edges and simply sort by the weight of them | 06:38 |
| ianw | http://logs.openstack.org/92/466592/1/check/gate-diskimage-builder-python27-ubuntu-xenial/0e9782d/console.html#_2017-05-22_05_14_36_368424 | 06:42 |
| *** adam_g has quit IRC | 06:42 | |
| ianw | interesting on the sort, it's very close | 06:42 |
| *** adam_g has joined #openstack-dib | 06:43 | |
| andreas-f | ianw: I don't say that it is not possible. This is just a list of changes that must be done. Not inheriting means to change the current concept. | 06:44 |
| andreas-f | But we should create a design that can be used by all plugins: we should not have something special for the partitioning / partition object. | 06:44 |
| ianw | andreas-f: actually, it barely requires changing. 466592 is basically working, modulo the sorting issues | 06:45 |
| ianw | 2017-05-22 16:44:14,959 DEBUG [diskimage_builder.block_device.tests.test_config] ['mount_mkfs_root', 'image0', 'fstab_mount_mkfs_root', 'mbr', 'root', 'mkfs_root'] | 06:46 |
| ianw | 2017-05-22 16:44:14,959 DEBUG [diskimage_builder.block_device.tests.test_config] ['image0', 'mbr', 'root', 'mkfs_root', 'mount_mkfs_root', 'fstab_mount_mkfs_root'] | 06:46 |
| ianw | the first one is from nx | 06:46 |
| ianw | i'm not sure why it gets different ordering than ours | 06:46 |
| ianw | that's for the following graph -> https://review.openstack.org/#/c/466591/1/diskimage_builder/block_device/tests/config/deep_graph.yaml | 06:46 |
| ianw | "A topological sort is a nonunique permutation of the nodes such that an edge from u to v implies that u appears before v in the topological sort order." | 06:53 |
| ianw | "nonunique permutation" | 06:53 |
| ianw | "Conversely, if a topological sort does not form a Hamiltonian path, the DAG will have two or more valid topological orderings, for in this case it is always possible to form a second valid ordering by swapping two consecutive vertices that are not connected by an edge to each other." | 06:55 |
| ianw | if our graphs have two valid topological orderings, that seems ... non deterministic | 06:55 |
| ianw | i think that's actually what i'm seeing, the order of nx.topological_sort() changes | 06:57 |
| ianw | ahh, hang on, missing and edge for some reason | 07:01 |
| *** andreas-f has quit IRC | 07:03 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at networkx for digraph implementation https://review.openstack.org/466592 | 07:36 |
| ianw | that one works ^ | 07:36 |
| ianw | yolanda: if you get time can you look at my comment in -> https://review.openstack.org/#/c/466592/2/diskimage_builder/block_device/level3/mount.py | 07:36 |
| ianw | why is the mount-point nodes not ordered from the config? re-ordering after seems wrong | 07:37 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: dhcp-all-interfaces.sh - Add support for InfiniBand interface DHCP https://review.openstack.org/465928 | 07:48 |
| *** Sukhdev has quit IRC | 07:50 | |
| ianw | yolanda: also, if you get a chance to add more configs to 466591 for the graph testing, well that would be great :) | 07:50 |
| *** adam_g has quit IRC | 08:54 | |
| *** adam_g has joined #openstack-dib | 08:55 | |
| *** chhavi has joined #openstack-dib | 08:55 | |
| *** adam_g has quit IRC | 09:54 | |
| *** adam_g has joined #openstack-dib | 09:55 | |
| *** adam_g has quit IRC | 10:09 | |
| *** adam_g has joined #openstack-dib | 10:10 | |
| *** hashar has joined #openstack-dib | 10:16 | |
| *** adam_g has quit IRC | 10:19 | |
| *** adam_g has joined #openstack-dib | 10:23 | |
| *** adam_g has quit IRC | 10:32 | |
| *** adam_g has joined #openstack-dib | 10:35 | |
| *** hashar has quit IRC | 10:44 | |
| *** adam_g has quit IRC | 10:48 | |
| *** adam_g has joined #openstack-dib | 10:49 | |
| *** chhavi has quit IRC | 11:20 | |
| *** lennyb_ has quit IRC | 11:24 | |
| *** lennyb has quit IRC | 11:24 | |
| *** lennyb has joined #openstack-dib | 11:25 | |
| *** lennyb has quit IRC | 11:27 | |
| *** lennyb has joined #openstack-dib | 11:28 | |
| *** lennyb_ has joined #openstack-dib | 11:29 | |
| *** lennyb has quit IRC | 11:29 | |
| *** lennyb_ has quit IRC | 11:29 | |
| *** lennyb has joined #openstack-dib | 11:30 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at networkx for digraph implementation https://review.openstack.org/466592 | 11:49 |
| ianw | andreas-f: ^ | 11:50 |
| ianw | andreas-f: https://review.openstack.org/#/c/466592/3/diskimage_builder/block_device/tests/test_config.py line 175 is a concrete example of using a DFS to walk the attached partition list below a "partitioning" object | 11:51 |
| ianw | as i mentioned in another comment, if ordering is important there, i think it should probably come from the graph itself. just chain the partitions in the order required and they'll be picked up by the parent in that order | 11:52 |
| ianw | you know, we could probably even do this as part of the graph generation | 11:54 |
| ianw | 1) generate topological_sort | 11:54 |
| ianw | 2) walk list, looking for "Partitioning" objects | 11:54 |
| ianw | 3) do DFS search on that node for "Partition" object children | 11:55 |
| ianw | 4) append all partition children to a variable in the "Partitioning" node | 11:55 |
| ianw | 5) all done. Partitioning.create() just walks it's self.partitions list, and the info it needs is there | 11:56 |
| ianw | that's basically what we have currently, just without all the magic recursive tree calling functions during config read | 11:57 |
| ianw | i don't know if i like that or not. + is that nodes don't have to be passed the whole graph. - is each config option that wants this will do something similar | 11:59 |
| *** chhavi has joined #openstack-dib | 12:39 | |
| *** aparnav has quit IRC | 13:08 | |
| *** mmedvede has quit IRC | 13:58 | |
| *** mmedvede has joined #openstack-dib | 14:01 | |
| *** hashar has joined #openstack-dib | 14:01 | |
| *** hashar has quit IRC | 14:25 | |
| *** tushar has joined #openstack-dib | 15:49 | |
| *** aparnav has joined #openstack-dib | 17:10 | |
| *** aparna has joined #openstack-dib | 17:12 | |
| *** aparnav has quit IRC | 17:16 | |
| *** aparna has quit IRC | 17:17 | |
| *** Sukhdev has joined #openstack-dib | 17:43 | |
| *** Sukhdev has quit IRC | 18:01 | |
| *** sauloaislan has joined #openstack-dib | 18:32 | |
| *** andreas-f has joined #openstack-dib | 19:07 | |
| *** chhavi has quit IRC | 19:48 | |
| *** pmannidi has quit IRC | 23:31 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at networkx for digraph implementation https://review.openstack.org/466592 | 23:37 |
| *** pmannidi has joined #openstack-dib | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!