pyspark.pipelines.create_sink#
- pyspark.pipelines.create_sink(name, format, options=None)[source]#
Creates a sink that can be targeted by streaming flows, providing a generic destination for flows to send data external to the pipeline.
- Parameters
name – The name of the sink.
format – The format of the sink, e.g. “parquet”.
options – A dict where the keys are the property names and the values are the property values. These properties will be set on the sink.