How do I check if any value is NaN in a Pandas DataFrame ?

How do I check if any value is NaN in a Pandas DataFrame ?

Asked on December 18, 2018 in Pandas.
Add Comment


  • 2 Answer(s)

    There are many options. Try anyone of the options below:

    import pandas as pd
    import numpy as np
     
    df = pd.DataFrame(np.random.randn(10,6))
    # Make a few areas have NaN values
    df.iloc[1:3,1] = np.nan
    df.iloc[5,3] = np.nan
    df.iloc[7:9,5] = np.nan
    

    Exectuing the above code, the dataframe is.,

              0       1           2         3        4           5
    0   0.520113  0.884000  1.260966 -0.236597  0.312972 -0.196281
    1  -0.837552  NaN       0.143017  0.862355  0.346550  0.842952
    2  -0.452595  NaN      -0.420790  0.456215  1.203459  0.527425
    3   0.317503 -0.917042  1.780938 -1.584102  0.432745  0.389797
    4  -0.722852  1.704820 -0.113821 -1.466458  0.083002  0.011722
    5  -0.622851 -0.251935 -1.498837  NaN       1.098323  0.273814
    6   0.329585  0.075312 -0.690209 -3.807924  0.489317 -0.841368
    7  -1.123433 -1.187496  1.868894 -2.046456 -0.949718  NaN
    8   1.133880 -0.110447  0.050385 -1.158387  0.188222  NaN
    9  -0.513741  1.196259  0.704537  0.982395 -0.585040 -1.693810
    

    1st option:

    The function df.isnull().any().any() returns a value that is a boolean.

    the funtion isnull() returns a DataFrame.,

          0    1      2     3     4    5
    0 False False False False False False
    1 False True  False False False False
    2 False True  False False False False
    3 False False False False False False
    4 False False False False False False
    5 False False False True  False False
    6 False False False False False False
    7 False False False False False True
    8 False False False False False True
    9 False False False False False False
    

    When the function df.isnull().any() is executed it returns the columns that have NaN values.,

    0 False
    1  True
    2 False
    3  True
    4 False
    5  True
    dtype: bool
    

    The function .any() will return the values that areTrue:

    > df.isnull().any().any()
    True
    

    2nd Option:

    The function df.isnull().sum().sum() will return a number indicating the total number of Nan values. The performance is as same as the function .any().any(). It gives the total number of NaN values in a column and the total number of values.,

    df.isnull().sum()
    0 0
    1 2
    2 0
    3 1
    4 0
    5 2
    dtype: int64
    

    At the end, to find the total number of  Nan values in the DataFrame execute the following code:

    df.isnull().sum().sum()
    5
    
    Answered on December 18, 2018.
    Add Comment

    Try the code below to find the NaN values in a specific column in a row:

    nan_rows = df[df['name column'].isnull()]
    
    Answered on December 18, 2018.
    Add Comment


  • Your Answer

    By posting your answer, you agree to the privacy policy and terms of service.