AttributeError: ‘DataFrame’ object has no attribute ‘map’

AttributeError: ‘DataFrame’ object has no attribute ‘map’

Asked on January 7, 2019 in Apache-spark.
Add Comment


  • 2 Answer(s)

    Here we cannot map a dataframe, but we can convert the dataframe to an RDD and map that by doing spark_df.rdd.map(). Prior to Spark 2.0, spark_df.map would alias to spark_df.rdd.map(). With Spark 2.0, you must explicitly call .rdd first.

    Answered on January 7, 2019.
    Add Comment

    You can’t map a dataframe, but you can convert the dataframe to an RDD and map that by doing spark_df.rdd.map(). Prior to Spark 2.0, spark_df.map would alias to spark_df.rdd.map(). With Spark 2.0, you must explicitly call .rdd first.

    Answered on March 5, 2019.
    Add Comment


  • Your Answer

    By posting your answer, you agree to the privacy policy and terms of service.