How do I run graphx with Python / pyspark ?

How do I run graphx with Python / pyspark ?

Add Comment


  • 3 Answer(s)

    Here this seems to be python bindings to GraphX are delayed at least to Spark 1.4 1.5 ∞. It is waiting behind the Java API.

    The status can be viewed here:  SPARK-3789 GRAPHX Python bindings for GraphX – ASF JIRA

    Answered on January 3, 2019.
    Add Comment

    In this GraphFrames can be referred in this link: https://github.com/graphframes/graphframes, which under the DataFrames API  GraphX algorithms is wrapped and it provides Python interface.

    This is working instance with slight modification  http://graphframes.github.io/quick-start.html,

    with the graphframes pkg loaded, start pyspark.

    pyspark –packages graphframes:graphframes:0.1.0-spark1.6

    The below is the python code:

    from graphframes import *
     
    # Create a Vertex DataFrame with unique ID column "id"
    v = sqlContext.createDataFrame([
      ("a", "Alice", 34),
      ("b", "Bob", 36),
      ("c", "Charlie", 30),
    ], ["id", "name", "age"])
     
    # Create an Edge DataFrame with "src" and "dst" columns
    e = sqlContext.createDataFrame([
      ("a", "b", "friend"),
      ("b", "c", "follow"),
      ("c", "b", "follow"),
    ], ["src", "dst", "relationship"])
    # Create a GraphFrame
    g = GraphFrame(v, e)
     
    # Query: Get in-degree of each vertex.
    g.inDegrees.show()
     
    # Query: Count the number of "follow" connections in the graph.
    g.edges.filter("relationship = 'follow'").count()
     
    # Run PageRank algorithm, and show results.
    results = g.pageRank(resetProbability=0.01, maxIter=20)
    results.vertices.select("id", "pagerank").show()
    
    Answered on January 3, 2019.
    Add Comment

    Here python API is not there in GraphX 0.9.0. This could be expected in future releases.

    Answered on January 3, 2019.
    Add Comment


  • Your Answer

    By posting your answer, you agree to the privacy policy and terms of service.