Viewing the content of a Spark Dataframe Column

Viewing the content of a Spark Dataframe Column

Asked on January 11, 2019 in Apache-spark.
Add Comment


  • 2 Answer(s)

    By underlying RDD, This can be accessed and map over it.

    df.rdd.map(lambda r: r.zip_code).collect()
    

    Here the select is used. By using Row objects output is wrapped:

    df.select('zip_code').collect()
    

    At the end, When there is need to inspect content:

    df.select('zip_code').show()
    

     

    Answered on January 11, 2019.
    Add Comment

    Here the full content can be seen by using below command:

    df.select("raw").take(1).foreach(println)
    

    show will display  an overview.

    Answered on January 11, 2019.
    Add Comment


  • Your Answer

    By posting your answer, you agree to the privacy policy and terms of service.