In this PySpark article, you have learned how to cast or change one DataFrame column Data Type to another type using withColumn(), selectExpr(), SQL. Tip: Also look at the CONVERT() function. This example is also available at GitHub for reference. The CAST() function converts a value (of any type) into a specified datatype. This function takes two arguments: the string to be converted, and the. withColumn("jobStartDate",col("jobStartDate").cast(DateType())) The most common way to convert a string to a numeric value is to use the CAST() function. This example uses the CAST expression to convert a decimal to an. 1) Converting a decimal to an integer example example. Let’s take some examples of using the CAST expression. To convert a String to INT uses sql conversion functions like cast or convert.
SQL CONVERT STRING TO INT CODE
withColumn("isGraduated",col("isGraduated").cast(BooleanType())) \ Code language: SQL (Structured Query Language) (sql) In this syntax, you specify the value and the data type to which you want to convert the value. Spark = ('').getOrCreate()įrom import StringType,BooleanType,DateTypeĭf2 = df.withColumn("age",col("age").cast(StringType())) \ CAST ( expression AS datatype ( length ) ) CONVERT ( datatype (. Complete Example of Casting PySpark Columnīelow is complete working example of how to convert the data types of DataFrame column. To convert a String to INT uses sql conversion functions like cast or convert. On SQL just wrap the column with the desired type you want.ĭf3.createOrReplaceTempView("CastExample")ĭf4 = spark.sql("SELECT STRING(age),BOOLEAN(isGraduated),DATE(jobStartDate) from CastExample")ĥ. In order to use on SQL, first, we need to create a table using createOrReplaceTempView(). We can also use PySpark SQL expression to change/cast the spark DataFrame column type. "cast(jobStartDate as string) jobStartDate") "cast(isGraduated as string) isGraduated", CAST ( '195' AS int ) CONVERT ( int, '225' ) The string to int conversion can be useful where you are taking user input and want to convert that into column’s data type before using. Both these functions are little different to use. SelectExpr() is a function in DataFrame which we can use to convert spark DataFrame column “age” from String to integer, “isGraduated” from boolean to string and “jobStartDate” from date to String.ĭf3 = df2.selectExpr("cast(age as int) age", You may use SQL CAST and CONVERT functions for converting int to string and vice versa. |- isGraduated: boolean (nullable = true) Dim MyInt, MyVar MyInt 4534 ' MyInt is an Integer. This example uses the CVar function to convert an expression to a Variant. MyString CStr(MyDouble) ' MyString contains '437.324'. Dim MyDouble, MyString MyDouble 437.324 ' MyDouble is a Double. Use withColumn() to convert the data type of a DataFrame column, This function takes column name you wanted to convert as a first argument and for the second argument apply the casting method cast() with DataType on the column. This example uses the CStr function to convert a numeric value to a String. |firstname|age|jobStartDate|isGraduated|gender|salary| |- jobStartDate: string (nullable = true)
0 kommentar(er)
