pyspark.sql.functions.any_value

pyspark.sql.functions.any_value(col, ignoreNulls=None)

Returns some value of col for a group of rows.

New in version 3.5.0.

Parameters

colColumn or column name

target column to work on.

ignoreNullsColumn or bool, optional

if first value is null then look for first non-null value.

Returns

Column

some value of col for a group of rows.

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame(
...     [(None, 1), ("a", 2), ("a", 3), ("b", 8), ("b", 2)], ["c1", "c2"])
>>> df.select(sf.any_value('c1'), sf.any_value('c2')).show()
+-------------+-------------+
|any_value(c1)|any_value(c2)|
+-------------+-------------+
|         NULL|            1|
+-------------+-------------+

>>> df.select(sf.any_value('c1', True), sf.any_value('c2', True)).show()
+-------------+-------------+
|any_value(c1)|any_value(c2)|
+-------------+-------------+
|            a|            1|
+-------------+-------------+