distinct

Restituisce un nuovo dataframe contenente le righe distinte in questo dataframe.

Sintassi

distinct()

Restituzioni

DataFrame: dataframe con record distinti.

Examples

df = spark.createDataFrame(
    [(14, "Tom"), (23, "Alice"), (23, "Alice")], ["age", "name"])
df.distinct().show()
# +---+-----+
# |age| name|
# +---+-----+
# | 14|  Tom|
# | 23|Alice|
# +---+-----+

df.distinct().count()
# 2

df = spark.createDataFrame(
    [(14, "Tom", "M"), (23, "Alice", "F"), (23, "Alice", "F"), (14, "Tom", "M")],
    ["age", "name", "gender"])
df.distinct().show()
# +---+-----+------+
# |age| name|gender|
# +---+-----+------+
# | 14|  Tom|     M|
# | 23|Alice|     F|
# +---+-----+------+