pyspark.sql.DataFrameWriter.jdbc#

DataFrameWriter.jdbc(url, table, mode=None, properties=None)[source]#

Saves the content of the DataFrame to an external database table via JDBC.

New in version 1.4.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters
tablestr

Name of the table in the external database.

modestr, optional

specifies the behavior of the save operation when data already exists.

  • append: Append contents of this DataFrame to existing data.

  • overwrite: Overwrite existing data.

  • ignore: Silently ignore this operation if data already exists.

  • error or errorifexists (default case): Throw an exception if data already exists.

propertiesdict

a dictionary of JDBC database connection arguments. Normally at least properties “user” and “password” with their corresponding values. For example { ‘user’ : ‘SYSTEM’, ‘password’ : ‘mypassword’ }

Other Parameters
Extra options

For the extra options, refer to Data Source Option for the version you use.

Notes

Don’t create too many partitions in parallel on a large cluster; otherwise Spark might crash your external database systems.