SatelliteGraph Details

Below you find usage examples and advanced configuration possibilities for SatelliteGraphs. The examples use arangosh and the @arangodb/satellite-graph module. You can also manage SatelliteGraphs via the HTTP API.

How to create a SatelliteGraph

SatelliteGraphs enforce and rely on special properties of the underlying collections and hence can only work with collections that are either created implicitly through the SatelliteGraph interface, or manually with the correct properties:

  • There needs to be a prototype collection with replicationFactor set to "satellite"
  • All other collections need to have distributeShardsLike set to the name of the prototype collection

Collections can be part of multiple SatelliteGraphs. This means that in contrast to SmartGraphs, SatelliteGraphs can be overlapping. If you have a larger SatelliteGraph and want to create an additional SatelliteGraph which only covers a part of it, then you can do so.

Create a graph

To create a SatelliteGraph in arangosh, use the satellite-graph module:

  1. arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
  2. arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
  3. arangosh> satelliteGraphModule._graph("satelliteGraph");

Show execution results

Hide execution results

  1. {[SatelliteGraph] 
  2. }

In contrast to General Graphs and SmartGraphs, you do not need to take care of the sharding and replication properties. The properties distributeShardsLike, replicationFactor and numberOfShards will be set automatically.

Add vertex collections

Adding vertex collections is analogous to General Graphs:

  1. arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
  2. arangosh> graph._addVertexCollection("aVertexCollection");

Show execution results

Hide execution results

If the collection "aVertexCollection" doesn’t exist yet, then the SatelliteGraph module will create it automatically with the correct properties. If it exists already, then its properties must be suitable for a SatelliteGraph (see prototype collection). Otherwise it will not be added.

Define relations

Adding edge collections works the same as with General Graphs, but again, the collections are created by the SatelliteGraph module with the right properties if they don’t exist already.

  1. arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
  2. arangosh> var relation = satelliteGraphModule._relation("isFriend", ["person"], ["person"]);
  3. arangosh> graph._extendEdgeDefinitions(relation);

Show execution results

Hide execution results

Existing edge collections can be added, but they require the distributeShardsLike property to reference the prototype collection.

The prototype collection

Every SatelliteGraph needs exactly one document collection with replicationFactor set to "satellite". This automatically leads to the collection having an exact amount of one shard per collection. This collection is selected as prototype.

All other collections of the SatelliteGraph need to inherit its properties by referencing its name in the distributeShardsLike property.

If collections are created implicitly through the SatelliteGraph module, then this is handled for you automatically. If you want to create the collections manually before adding them to the SatelliteGraph, then you need to take care of these properties.

Prototype collection examples

Creating an empty SatelliteGraph: No prototype collection is present.

  1. arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
  2. arangosh> satelliteGraphModule._create("satelliteGraph");

Show execution results

Hide execution results

  1. {[SatelliteGraph] 
  2. }

Creating an empty SatelliteGraph, then adding a document (vertex) collection. This leads to the creation of a prototype collection "myPrototypeColl" (assuming that no collection with this name existed before):

  1. arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
  2. arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
  3. arangosh> graph._addVertexCollection("myPrototypeColl");

Show execution results

Hide execution results

Creating an empty SatelliteGraph, then adding an edge definition. This will select the collection "person" as prototype collection, as it is the only document (vertex) collection. If you supply more than one document collection, then one of the collections will be chosen arbitrarily as prototype collection.

  1. arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
  2. arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
  3. arangosh> var relation = satelliteGraphModule._relation("isFriend", ["person"], ["person"]);
  4. arangosh> graph._extendEdgeDefinitions(relation);

Show execution results

Hide execution results

The prototype collection can and also will be automatically selected during the graph creation process if at least one document (vertex) collection is supplied directly. If more then one are available, they will be chosen randomly as well, regardless whether they are set inside the edge definition itself or set as a vertex/orphan collection.

Utilizing SatelliteGraphs

Obviously, a SatelliteGraph must be created before it can be queried. Valid operations that can then be optimized are (k-)shortest path(s) computations and traversals. Both also allow for combination with local joins or other SatelliteGraph operations.

Here is an example showing the difference between the execution of a General Graph and a SatelliteGraph traversal query:

  1. First we setup our graphs and collections.
  1. arangosh> var graphModule = require("@arangodb/general-graph");
  2. arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
  3. arangosh> graphModule._create("normalGraph", [ graphModule._relation("edges", "vertices", "vertices") ], [], {});
  4. arangosh> satelliteGraphModule._create("satelliteGraph", [ satelliteGraphModule._relation("satEdges", "satVertices", "satVertices") ], [], {});
  5. arangosh> db._create("collection", {numberOfShards: 8});

Show execution results

Hide execution results

  1. {[Graph] 
  2.   "edges" : [ArangoCollection 10073, "edges" (type edge, status loaded)], 
  3.   "vertices" : [ArangoCollection 10072, "vertices" (type document, status loaded)] 
  4. }
  5. {[SatelliteGraph] 
  6.   "satEdges" : [ArangoCollection 10079, "satEdges" (type edge, status loaded)], 
  7.   "satVertices" : [ArangoCollection 10077, "satVertices" (type document, status loaded)] 
  8. }
  9. [ArangoCollection 10081, "collection" (type document, status loaded)]
  1. Let us analyze a query involving a traversal:
  1. arangosh> db._explain(`FOR doc in collection FOR v,e,p IN OUTBOUND "vertices/start" GRAPH "normalGraph" RETURN [doc,v,e,p]`, {}, {colors: false});

Show execution results

Hide execution results

  1. Query String (99 chars, cacheable: true):
  2.  FOR doc in collection FOR v,e,p IN OUTBOUND "vertices/start" GRAPH "normalGraph" RETURN [doc,v,e,p]
  4. Execution plan:
  5.  Id   NodeType                  Site  Est.   Comment
  6.   1   SingletonNode             DBS      1   * ROOT
  7.   2   EnumerateCollectionNode   DBS      0     - FOR doc IN collection   /* full collection scan, 8 shard(s) */
  8.   8   RemoteNode                COOR     0       - REMOTE
  9.   9   GatherNode                COOR     0       - GATHER   /* unsorted */
  10.   3   TraversalNode             COOR     1       - FOR v  /* vertex */, e  /* edge */, p  /* paths: vertices, edges */ IN 1..1  /* min..maxPathDepth */ OUTBOUND 'vertices/start' /* startnode */  GRAPH 'normalGraph'
  11.   4   CalculationNode           COOR     1         - LET #6 = [ doc, v, e, p ]   /* simple expression */   /* collections used: doc : collection */
  12.   5   ReturnNode                COOR     1         - RETURN #6
  14. Indexes used:
  15.  By   Name   Type   Collection   Unique   Sparse   Selectivity   Fields        Ranges
  16.   3   edge   edge   edges        false    false       100.00 %   [ `_from` ]   base OUTBOUND
  18. Traversals on graphs:
  19.  Id  Depth  Vertex collections  Edge collections  Options                                  Filter / Prune Conditions
  20.  3   1..1   vertices            edges             uniqueVertices: none, uniqueEdges: path                           
  22. Optimization rules applied:
  23.  Id   RuleName
  24.   1   scatter-in-cluster
  25.   2   remove-unnecessary-remote-scatter
  27. Optimization rules with highest execution times:
  28.  RuleName                                        Duration [s]
  29.  scatter-in-cluster                                   0.00001
  30.  remove-unnecessary-remote-scatter                    0.00001
  31.  restrict-to-single-shard                             0.00001
  32.  optimize-traversals                                  0.00000
  33.  use-indexes                                          0.00000
  35. 59 rule(s) executed, 1 plan(s) created

You can see that the TraversalNode is executed on a Coordinator, and only the EnumerateCollectionNode is executed on DB-Servers. This will happen for each of the 8 shards in collection.

  1. Let us now have a look at the same query using a SatelliteGraph:
  1. arangosh> db._explain(`FOR doc in collection FOR v,e,p IN OUTBOUND "vertices/start" GRAPH "satelliteGraph" RETURN [doc,v,e,p]`, {}, {colors: false});

Show execution results

Hide execution results

  1. Query String (102 chars, cacheable: true):
  2.  FOR doc in collection FOR v,e,p IN OUTBOUND "vertices/start" GRAPH "satelliteGraph" RETURN 
  3.  [doc,v,e,p]
  5. Execution plan:
  6.  Id   NodeType                  Site  Est.   Comment
  7.   1   SingletonNode             DBS      1   * ROOT
  8.   2   EnumerateCollectionNode   DBS      0     - FOR doc IN collection   /* full collection scan, 8 shard(s) */
  9.  10   TraversalNode             DBS      1       - FOR v  /* vertex */, e  /* edge */, p  /* paths: vertices, edges */ IN 1..1  /* min..maxPathDepth */ OUTBOUND 'vertices/start' /* startnode */  satEdges /* local graph node, used as satellite */
  10.   4   CalculationNode           DBS      1         - LET #6 = [ doc, v, e, p ]   /* simple expression */   /* collections used: doc : collection */
  11.  13   RemoteNode                COOR     1         - REMOTE
  12.  14   GatherNode                COOR     1         - GATHER   /* parallel, unsorted */
  13.   5   ReturnNode                COOR     1         - RETURN #6
  15. Indexes used:
  16.  By   Name   Type   Collection   Unique   Sparse   Selectivity   Fields        Ranges
  17.  10   edge   edge   satEdges     false    false       100.00 %   [ `_from` ]   base OUTBOUND
  19. Traversals on graphs:
  20.  Id  Depth  Vertex collections  Edge collections  Options                                  Filter / Prune Conditions
  21.  10  1..1   satVertices         satEdges          uniqueVertices: none, uniqueEdges: path                           
  23. Optimization rules applied:
  24.  Id   RuleName
  25.   1   scatter-in-cluster
  26.   2   scatter-satellite-graphs
  27.   3   remove-satellite-joins
  28.   4   distribute-filtercalc-to-cluster
  29.   5   remove-unnecessary-remote-scatter
  30.   6   parallelize-gather
  32. Optimization rules with highest execution times:
  33.  RuleName                                        Duration [s]
  34.  scatter-satellite-graphs                             0.00002
  35.  scatter-in-cluster                                   0.00001
  36.  remove-satellite-joins                               0.00001
  37.  parallelize-gather                                   0.00001
  38.  use-indexes                                          0.00001
  40. 59 rule(s) executed, 1 plan(s) created

Note that now the TraversalNode is executed on each DB-Server, leading to a great reduction in required network communication, and hence potential gains in query performance.

Convert General Graphs or SmartGraphs to SatelliteGraphs

If you want to transform an existing General Graph or SmartGraph into a SatelliteGraph, then you need to dump and restore your previous graph. This is necessary for the initial data replication and because some collection properties are immutable.

Use arangodump and arangorestore. The only thing you have to change in this pipeline is that you create the new collections during creation with the SatelliteGraph module or add collections manually to the SatelliteGraph before starting the arangorestore process.