layout: post
title: “Using F# for database related tasks”
description: “Twenty six low-risk ways to use F# at work (part 4)”
categories: []
seriesId: “Low-risk ways to use F# at work”
seriesOrder: 4


This post is a continuation of the previous series on low-risk and incremental ways to use F# at work.

In this one, we’ll see how F# can be unexpectedly helpful when it comes to database related tasks.

Series contents

Before moving on to the content of the post, here’s the full list of the twenty six ways:

Part 1 - Using F# to explore and develop interactively

1. Use F# to explore the .NET framework interactively

2. Use F# to test your own code interactively

3. Use F# to play with webservices interactively

4. Use F# to play with UI’s interactively

Part 2 - Using F# for development and devops scripts

5. Use FAKE for build and CI scripts

6. An F# script to check that a website is responding

7. An F# script to convert an RSS feed into CSV

8. An F# script that uses WMI to check the stats of a process

9. Use F# for configuring and managing the cloud

Part 3 - Using F# for testing

10. Use F# to write unit tests with readable names

11. Use F# to run unit tests programmatically

12. Use F# to learn to write unit tests in other ways

13. Use FsCheck to write better unit tests

14. Use FsCheck to create random dummy data

15. Use F# to create mocks

16. Use F# to do automated browser testing

17. Use F# for Behaviour Driven Development

Part 4. Using F# for database related tasks

18. Use F# to replace LINQpad

19. Use F# to unit test stored procedures

20. Use FsCheck to generate random database records

21. Use F# to do simple ETL

22. Use F# to generate SQL Agent scripts

Part 5: Other interesting ways of using F#

23. Use F# for parsing

24. Use F# for diagramming and visualization

25. Use F# for accessing web-based data stores

26. Use F# for data science and machine learning

(BONUS) 27: Balance the generation schedule for the UK power station fleet


This next group of suggestions is all about working with databases, and MS SQL Server in particular.

Relational databases are a critical part of most applications, but most teams do not approach the management of these in the same way as with other development tasks.

For example, how many teams do you know that unit test their stored procedures?

Or their ETL jobs?

Or generate T-SQL admin scripts and other boilerplate using a non-SQL scripting language that’s stored in source control?

Here’s where F# can shine over other scripting languages, and even over T-SQL itself.

  • The database type providers in F# give you the power to create simple, short scripts for testing and admin, with the bonus that…
  • The scripts are type-checked and will fail at compile time if the database schema changes, which means that…
  • The whole process works really well with builds and continuous integration processes, which in turn means that…
  • You have really high confidence in your database related code!

We’ll look at a few examples to demonstrate what I’m talking about:

  • Unit testing stored procedures
  • Using FsCheck to generate random records
  • Doing simple ETL with F#
  • Generating SQL Agent scripts

Getting set up

The code for this section is available on github.
In there, there are some SQL scripts to create the sample database, tables and stored procs that I’ll use in these examples.

To run the examples, then, you’ll need SQL Express or SQL Server running locally or somewhere accessible, with the relevant setup scripts having been run.

Which type provider?

There are a number of SQL Type Providers for F# — see the fsharp.org Data Access page. For these examples, I’m going to use
the SqlDataConnection type provider, which is part of the FSharp.Data.TypeProviders DLL.
It uses SqlMetal behind the scenes and so only works with SQL Server databases.

The SQLProvider project is another good choice — it supports MySql, SQLite and other non-Microsoft databases.

18. Use F# to replace LINQPad

The code for this section is available on github.

LINQPad is a great tool for doing queries against databases, and is also a general scratchpad for C#/VB/F# code.

You can use F# interactive to do many of the same things — you get queries, autocompletion, etc., just like LINQPad.

For example, here’s one that counts customers with a certain email domain.

  1. [<Literal>]
  2. let connectionString = "Data Source=localhost; Initial Catalog=SqlInFsharp; Integrated Security=True;"
  3. type Sql = SqlDataConnection<connectionString>
  4. let db = Sql.GetDataContext()
  5. // find the number of customers with a gmail domain
  6. query {
  7. for c in db.Customer do
  8. where (c.Email.EndsWith("gmail.com"))
  9. select c
  10. count
  11. }

If you want to see what SQL code is generated, you can turn logging on, of course:

  1. // optional, turn logging on
  2. db.DataContext.Log <- Console.Out

The logged output for this query is:

  1. SELECT COUNT(*) AS [value]
  2. FROM [dbo].[Customer] AS [t0]
  3. WHERE [t0].[Email] LIKE @p0
  4. -- @p0: Input VarChar (Size = 8000; Prec = 0; Scale = 0) [%gmail.com]

You can also do more complicated things, such as using subqueries. Here’s an example from MSDN:

Note that, as befitting a functional approach, queries are nice and composable.

  1. // Find students who have signed up at least one course.
  2. query {
  3. for student in db.Student do
  4. where (query { for courseSelection in db.CourseSelection do
  5. exists (courseSelection.StudentID = student.StudentID) })
  6. select student
  7. }

And if the SQL engine doesn’t support certain functions such as regexes, and assuming the size of the data is not too large,
you can just stream the data out and do the processing in F#.

  1. // find the most popular domain for people born in each decade
  2. let getDomain email =
  3. Regex.Match(email,".*@(.*)").Groups.[1].Value
  4. let getDecade (birthdate:Nullable<DateTime>) =
  5. if birthdate.HasValue then
  6. birthdate.Value.Year / 10 * 10 |> Some
  7. else
  8. None
  9. let topDomain list =
  10. list
  11. |> Seq.distinct
  12. |> Seq.head
  13. |> snd
  14. db.Customer
  15. |> Seq.map (fun c -> getDecade c.Birthdate, getDomain c.Email)
  16. |> Seq.groupBy fst
  17. |> Seq.sortBy fst
  18. |> Seq.map (fun (decade, group) -> (decade,topDomain group))
  19. |> Seq.iter (printfn "%A")

As you can see from the code above, the nice thing about doing the processing in F# is that you can define helper functions separately and connect them together easily.

19. Use F# to unit test stored procedures

The code for this section is available on github.

Now let’s look at how we can use the type provider to make creating unit tests for stored procs really easy.

First, I create a helper module (which I’ll call DbLib) to set up the connection and to provide shared utility functions such as resetDatabase,
which will be called before each test.

  1. module DbLib
  2. [<Literal>]
  3. let connectionString = "Data Source=localhost; Initial Catalog=SqlInFsharp;Integrated Security=True;"
  4. type Sql = SqlDataConnection<connectionString>
  5. let removeExistingData (db:DbContext) =
  6. let truncateTable name =
  7. sprintf "TRUNCATE TABLE %s" name
  8. |> db.DataContext.ExecuteCommand
  9. |> ignore
  10. ["Customer"; "CustomerImport"]
  11. |> List.iter truncateTable
  12. let insertReferenceData (db:DbContext) =
  13. [ "US","United States";
  14. "GB","United Kingdom" ]
  15. |> List.iter (fun (code,name) ->
  16. let c = new Sql.ServiceTypes.Country()
  17. c.IsoCode <- code; c.CountryName <- name
  18. db.Country.InsertOnSubmit c
  19. )
  20. db.DataContext.SubmitChanges()
  21. // removes all data and restores db to known starting point
  22. let resetDatabase() =
  23. use db = Sql.GetDataContext()
  24. removeExistingData db
  25. insertReferenceData db

Now I can write a unit test, using NUnit say, just like any other unit test.

Assume that we have Customer table, and a sproc called up_Customer_Upsert that either inserts a new customer or updates an existing one, depending on whether
the passed in customer id is null or not.

Here’s what a test looks like:

  1. [<Test>]
  2. let ``When upsert customer called with null id, expect customer created with new id``() =
  3. DbLib.resetDatabase()
  4. use db = DbLib.Sql.GetDataContext()
  5. // create customer
  6. let newId = db.Up_Customer_Upsert(Nullable(),"Alice","x@example.com",Nullable())
  7. // check new id
  8. Assert.Greater(newId,0)
  9. // check one customer exists
  10. let customerCount = db.Customer |> Seq.length
  11. Assert.AreEqual(1,customerCount)

Note that, because the setup is expensive, I do multiple asserts in the test. This could be refactored if you find this too ugly!

Here’s one that tests that updates work:

  1. [<Test>]
  2. let ``When upsert customer called with existing id, expect customer updated``() =
  3. DbLib.resetDatabase()
  4. use db = DbLib.Sql.GetDataContext()
  5. // create customer
  6. let custId = db.Up_Customer_Upsert(Nullable(),"Alice","x@example.com",Nullable())
  7. // update customer
  8. let newId = db.Up_Customer_Upsert(Nullable custId,"Bob","y@example.com",Nullable())
  9. // check id hasnt changed
  10. Assert.AreEqual(custId,newId)
  11. // check still only one customer
  12. let customerCount = db.Customer |> Seq.length
  13. Assert.AreEqual(1,customerCount)
  14. // check customer columns are updated
  15. let customer = db.Customer |> Seq.head
  16. Assert.AreEqual("Bob",customer.Name)

And one more, that checks for exceptions:

  1. [<Test>]
  2. let ``When upsert customer called with blank name, expect validation error``() =
  3. DbLib.resetDatabase()
  4. use db = DbLib.Sql.GetDataContext()
  5. try
  6. // try to create customer will a blank name
  7. db.Up_Customer_Upsert(Nullable(),"","x@example.com",Nullable()) |> ignore
  8. Assert.Fail("expecting a SqlException")
  9. with
  10. | :? System.Data.SqlClient.SqlException as ex ->
  11. Assert.That(ex.Message,Is.StringContaining("@Name"))
  12. Assert.That(ex.Message,Is.StringContaining("blank"))

As you can see, the whole process is very straightforward.

These tests can be compiled and run as part of the continuous integration scripts.
And what is great is that, if the database schema gets out of sync with the code, then the tests will fail to even compile!

20. Use FsCheck to generate random database records

The code for this section is available on github.

As I showed in an earlier example, you can use FsCheck to generate random data. In this case we’ll use it to generate random
records in the database.

Let’s say we have a CustomerImport table, defined as below. (We’ll use this table in the next section on ETL)

  1. CREATE TABLE dbo.CustomerImport (
  2. CustomerId int NOT NULL IDENTITY(1,1)
  3. ,FirstName varchar(50) NOT NULL
  4. ,LastName varchar(50) NOT NULL
  5. ,EmailAddress varchar(50) NOT NULL
  6. ,Age int NULL
  7. CONSTRAINT PK_CustomerImport PRIMARY KEY CLUSTERED (CustomerId)
  8. )

Using the same code as before, we can then generate random instances of CustomerImport.

  1. [<Literal>]
  2. let connectionString = "Data Source=localhost; Initial Catalog=SqlInFsharp; Integrated Security=True;"
  3. type Sql = SqlDataConnection<connectionString>
  4. // a list of names to sample
  5. let possibleFirstNames =
  6. ["Merissa";"Kenneth";"Zora";"Oren"]
  7. let possibleLastNames =
  8. ["Applewhite";"Feliz";"Abdulla";"Strunk"]
  9. // generate a random name by picking from the list at random
  10. let generateFirstName() =
  11. FsCheck.Gen.elements possibleFirstNames
  12. let generateLastName() =
  13. FsCheck.Gen.elements possibleLastNames
  14. // generate a random email address by combining random users and domains
  15. let generateEmail() =
  16. let userGen = FsCheck.Gen.elements ["a"; "b"; "c"; "d"; "e"; "f"]
  17. let domainGen = FsCheck.Gen.elements ["gmail.com"; "example.com"; "outlook.com"]
  18. let makeEmail u d = sprintf "%s@%s" u d
  19. FsCheck.Gen.map2 makeEmail userGen domainGen

So far so good.

Now we get to the age column, which is nullable. This means we can’t generate random ints, but instead
we have to generate random Nullable<int>s. This is where type checking is really useful — the compiler has forced us to take that into account.
So to make sure we cover all the bases, we’ll generate a null value one time out of twenty.

  1. // Generate a random nullable age.
  2. // Note that because age is nullable,
  3. // the compiler forces us to take that into account
  4. let generateAge() =
  5. let nonNullAgeGenerator =
  6. FsCheck.Gen.choose(1,99)
  7. |> FsCheck.Gen.map (fun age -> Nullable age)
  8. let nullAgeGenerator =
  9. FsCheck.Gen.constant (Nullable())
  10. // 19 out of 20 times choose a non null age
  11. FsCheck.Gen.frequency [
  12. (19,nonNullAgeGenerator)
  13. (1,nullAgeGenerator)
  14. ]

Putting it altogether…

  1. // a function to create a customer
  2. let createCustomerImport first last email age =
  3. let c = new Sql.ServiceTypes.CustomerImport()
  4. c.FirstName <- first
  5. c.LastName <- last
  6. c.EmailAddress <- email
  7. c.Age <- age
  8. c //return new record
  9. // use applicatives to create a customer generator
  10. let generateCustomerImport =
  11. createCustomerImport
  12. <!> generateFirstName()
  13. <*> generateLastName()
  14. <*> generateEmail()
  15. <*> generateAge()

Once we have a random generator, we can fetch as many records as we like, and insert them using the type provider.

In the code below, we’ll generate 10,000 records, hitting the database in batches of 1,000 records.

  1. let insertAll() =
  2. use db = Sql.GetDataContext()
  3. // optional, turn logging on or off
  4. // db.DataContext.Log <- Console.Out
  5. // db.DataContext.Log <- null
  6. let insertOne counter customer =
  7. db.CustomerImport.InsertOnSubmit customer
  8. // do in batches of 1000
  9. if counter % 1000 = 0 then
  10. db.DataContext.SubmitChanges()
  11. // generate the records
  12. let count = 10000
  13. let generator = FsCheck.Gen.sample 0 count generateCustomerImport
  14. // insert the records
  15. generator |> List.iteri insertOne
  16. db.DataContext.SubmitChanges() // commit any remaining

Finally, let’s do it and time it.

  1. #time
  2. insertAll()
  3. #time

It’s not as fast as using BCP, but it is plenty adequate for testing. For example, it only takes a few seconds to create the 10,000 records above.

I want to stress that this is a single standalone script, not a heavy binary, so it is really easy to tweak and run on demand.

And of course you get all the goodness of a scripted approach, such as being able to store it in source control, track changes, etc.

21. Use F# to do simple ETL

The code for this section is available on github.

Say that you need to transfer data from one table to another, but it is not a totally straightforward copy,
as you need to do some mapping and transformation.

This is a classic ETL (Extract/Transform/Load) situation, and most people will reach for SSIS.

But for some situations, such as one off imports, and where the volumes are not large, you could use F# instead. Let’s have a look.

Say that we are importing data into a master table that looks like this:

  1. CREATE TABLE dbo.Customer (
  2. CustomerId int NOT NULL IDENTITY(1,1)
  3. ,Name varchar(50) NOT NULL
  4. ,Email varchar(50) NOT NULL
  5. ,Birthdate datetime NULL
  6. )

But the system we’re importing from has a different format, like this:

  1. CREATE TABLE dbo.CustomerImport (
  2. CustomerId int NOT NULL IDENTITY(1,1)
  3. ,FirstName varchar(50) NOT NULL
  4. ,LastName varchar(50) NOT NULL
  5. ,EmailAddress varchar(50) NOT NULL
  6. ,Age int NULL
  7. )

As part of this import then, we’re going to have to:

  • Concatenate the FirstName and LastName columns into one Name column
  • Map the EmailAddress column to the Email column
  • Calculate a Birthdate given an Age
  • I’m going to skip the CustomerId for now — hopefully we aren’t using IDENTITY columns in practice.

The first step is to define a function that maps source records to target records. In this case, we’ll call it makeTargetCustomer.

Here’s some code for this:

  1. [<Literal>]
  2. let sourceConnectionString =
  3. "Data Source=localhost; Initial Catalog=SqlInFsharp; Integrated Security=True;"
  4. [<Literal>]
  5. let targetConnectionString =
  6. "Data Source=localhost; Initial Catalog=SqlInFsharp; Integrated Security=True;"
  7. type SourceSql = SqlDataConnection<sourceConnectionString>
  8. type TargetSql = SqlDataConnection<targetConnectionString>
  9. let makeName first last =
  10. sprintf "%s %s" first last
  11. let makeBirthdate (age:Nullable<int>) =
  12. if age.HasValue then
  13. Nullable (DateTime.Today.AddYears(-age.Value))
  14. else
  15. Nullable()
  16. let makeTargetCustomer (sourceCustomer:SourceSql.ServiceTypes.CustomerImport) =
  17. let targetCustomer = new TargetSql.ServiceTypes.Customer()
  18. targetCustomer.Name <- makeName sourceCustomer.FirstName sourceCustomer.LastName
  19. targetCustomer.Email <- sourceCustomer.EmailAddress
  20. targetCustomer.Birthdate <- makeBirthdate sourceCustomer.Age
  21. targetCustomer // return it

With this transform in place, the rest of the code is easy, we just just read from the source and write to the target.

  1. let transferAll() =
  2. use sourceDb = SourceSql.GetDataContext()
  3. use targetDb = TargetSql.GetDataContext()
  4. let insertOne counter customer =
  5. targetDb.Customer.InsertOnSubmit customer
  6. // do in batches of 1000
  7. if counter % 1000 = 0 then
  8. targetDb.DataContext.SubmitChanges()
  9. printfn "...%i records transferred" counter
  10. // get the sequence of source records
  11. sourceDb.CustomerImport
  12. // transform to a target record
  13. |> Seq.map makeTargetCustomer
  14. // and insert
  15. |> Seq.iteri insertOne
  16. targetDb.DataContext.SubmitChanges() // commit any remaining
  17. printfn "Done"

Because these are sequence operations, only one record at a time is in memory (excepting the LINQ submit buffer), so even large data sets can
be processed.

To see it in use, first insert a number of records using the dummy data script just discussed, and then run the transfer as follows:

  1. #time
  2. transferAll()
  3. #time

Again, it only takes a few seconds to transfer 10,000 records.

And again, this is a single standalone script — it’s a very lightweight way to create simple ETL jobs.

22. Use F# to generate SQL Agent scripts

For the last database related suggestion, let me suggest the idea of generating SQL Agent scripts from code.

In any decent sized shop you may have hundreds or thousands of SQL Agent jobs. In my opinion, these should all be stored as script files, and
loaded into the database when provisioning/building the system.

Alas, there are often subtle differences between dev, test and production environments: connection strings, authorization, alerts, log configuration, etc.

That naturally leads to the problem of trying to keep three different copies of a script around, which in turn makes you think:
why not have one script and parameterize it for the environment?

But now you are dealing with lots of ugly SQL code! The scripts that create SQL agent jobs are typically hundreds of lines long and were not really designed
to be maintained by hand.

F# to the rescue!

In F#, it’s really easy to create some simple record types that store all the data you need to generate and configure a job.

For example, in the script below:

  • I created a union type called Step that could store a Package, Executable, Powershell and so on.
  • Each of these step types in turn have their own specific properties, so that a Package has a name and variables, and so on.
  • A JobInfo consists of a name plus a list of Steps.
  • An agent script is generated from a JobInfo plus a set of global properties associated with an environment, such as the databases, shared folder locations, etc.
  1. let thisDir = __SOURCE_DIRECTORY__
  2. System.IO.Directory.SetCurrentDirectory (thisDir)
  3. #load @"..\..\SqlAgentLibrary.Lib.fsx"
  4. module MySqlAgentJob =
  5. open SqlAgentLibrary.Lib.SqlAgentLibrary
  6. let PackageFolder = @"\shared\etl\MyJob"
  7. let step1 = Package {
  8. Name = "An SSIS package"
  9. Package = "AnSsisPackage.dtsx"
  10. Variables =
  11. [
  12. "EtlServer", "EtlServer"
  13. "EtlDatabase", "EtlDatabase"
  14. "SsisLogServer", "SsisLogServer"
  15. "SsisLogDatabase", "SsisLogDatabase"
  16. ]
  17. }
  18. let step2 = Package {
  19. Name = "Another SSIS package"
  20. Package = "AnotherSsisPackage.dtsx"
  21. Variables =
  22. [
  23. "EtlServer", "EtlServer2"
  24. "EtlDatabase", "EtlDatabase2"
  25. "SsisLogServer", "SsisLogServer2"
  26. "SsisLogDatabase", "SsisLogDatabase2"
  27. ]
  28. }
  29. let jobInfo = {
  30. JobName = "My SqlAgent Job"
  31. JobDescription = "Copy data from one place to another"
  32. JobCategory = "ETL"
  33. Steps =
  34. [
  35. step1
  36. step2
  37. ]
  38. StepsThatContinueOnFailure = []
  39. JobSchedule = None
  40. JobAlert = None
  41. JobNotification = None
  42. }
  43. let generate globals =
  44. writeAgentScript globals jobInfo
  45. module DevEnvironment =
  46. let globals =
  47. [
  48. // global
  49. "Environment", "DEV"
  50. "PackageFolder", @"\shared\etl\MyJob"
  51. "JobServer", "(local)"
  52. // General variables
  53. "JobName", "Some packages"
  54. "SetStartFlag", "2"
  55. "SetEndFlag", "0"
  56. // databases
  57. "Database", "mydatabase"
  58. "Server", "localhost"
  59. "EtlServer", "localhost"
  60. "EtlDatabase", "etl_config"
  61. "SsisLogServer", "localhost"
  62. "SsisLogDatabase", "etl_config"
  63. ] |> Map.ofList
  64. let generateJob() =
  65. MySqlAgentJob.generate globals
  66. DevEnvironment.generateJob()

I can’t share the actual F# code, but I think you get the idea. It’s quite simple to create.

Once we have these .FSX files, we can generate the real SQL Agent scripts en-masse and then deploy them to the appropriate servers.

Below is an example of a SQL Agent script that might be generated automatically from the .FSX file.

As you can see, it is a nicely laid out and formatted T-SQL script. The idea is that a DBA can review it and be confident that no magic is happening, and thus be
willing to accept it as input.

On the other hand, it would be risky to maintain scripts like. Editing the SQL code directly could be risky.
Better to use type-checked (and more concise) F# code than untyped T-SQL!

  1. USE [msdb]
  2. GO
  3. -- =====================================================
  4. -- Script that deletes and recreates the SQL Agent job 'My SqlAgent Job'
  5. --
  6. -- The job steps are:
  7. -- 1) An SSIS package
  8. -- {Continue on error=false}
  9. -- 2) Another SSIS package
  10. -- {Continue on error=false}
  11. -- =====================================================
  12. -- =====================================================
  13. -- Environment is DEV
  14. --
  15. -- The other global variables are:
  16. -- Database = mydatabase
  17. -- EtlDatabase = etl_config
  18. -- EtlServer = localhost
  19. -- JobName = My SqlAgent Job
  20. -- JobServer = (local)
  21. -- PackageFolder = \\shared\etl\MyJob\
  22. -- Server = localhost
  23. -- SetEndFlag = 0
  24. -- SetStartFlag = 2
  25. -- SsisLogDatabase = etl_config
  26. -- SsisLogServer = localhost
  27. -- =====================================================
  28. -- =====================================================
  29. -- Create job
  30. -- =====================================================
  31. -- ---------------------------------------------
  32. -- Delete Job if it exists
  33. -- ---------------------------------------------
  34. IF EXISTS (SELECT job_id FROM msdb.dbo.sysjobs_view WHERE name = 'My SqlAgent Job')
  35. BEGIN
  36. PRINT 'Deleting job "My SqlAgent Job"'
  37. EXEC msdb.dbo.sp_delete_job @job_name='My SqlAgent Job', @delete_unused_schedule=0
  38. END
  39. -- ---------------------------------------------
  40. -- Create Job
  41. -- ---------------------------------------------
  42. BEGIN TRANSACTION
  43. DECLARE @ReturnCode INT
  44. SELECT @ReturnCode = 0
  45. -- ---------------------------------------------
  46. -- Create Category if needed
  47. -- ---------------------------------------------
  48. IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name='ETL' AND category_class=1)
  49. BEGIN
  50. PRINT 'Creating category "ETL"'
  51. EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name='ETL'
  52. IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
  53. END
  54. -- ---------------------------------------------
  55. -- Create Job
  56. -- ---------------------------------------------
  57. DECLARE @jobId BINARY(16)
  58. PRINT 'Creating job "My SqlAgent Job"'
  59. EXEC @ReturnCode = msdb.dbo.sp_add_job @job_name='My SqlAgent Job',
  60. @enabled=1,
  61. @category_name='ETL',
  62. @owner_login_name=N'sa',
  63. @description='Copy data from one place to another',
  64. @job_id = @jobId OUTPUT
  65. IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
  66. PRINT '-- ---------------------------------------------'
  67. PRINT 'Create step 1: "An SSIS package"'
  68. PRINT '-- ---------------------------------------------'
  69. DECLARE @Step1_Name nvarchar(50) = 'An SSIS package'
  70. DECLARE @Step1_Package nvarchar(170) = 'AnSsisPackage.dtsx'
  71. DECLARE @Step1_Command nvarchar(1700) =
  72. '/FILE "\\shared\etl\MyJob\AnSsisPackage.dtsx"' +
  73. ' /CHECKPOINTING OFF' +
  74. ' /SET "\Package.Variables[User::SetFlag].Value";"2"' +
  75. ' /SET "\Package.Variables[User::JobName].Value";""' +
  76. ' /SET "\Package.Variables[User::SourceServer].Value";"localhost"' +
  77. ' /SET "\Package.Variables[User::SourceDatabaseName].Value";"etl_config"' +
  78. ' /REPORTING E'
  79. EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=@Step1_Name,
  80. @step_id=1,
  81. @on_success_action=3,
  82. @on_fail_action=2,
  83. @subsystem=N'SSIS',
  84. @command=@Step1_Command
  85. IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
  86. PRINT '-- ---------------------------------------------'
  87. PRINT 'Create step 2: "Another SSIS Package"'
  88. PRINT '-- ---------------------------------------------'
  89. DECLARE @Step2_Name nvarchar(50) = 'Another SSIS Package'
  90. DECLARE @Step2_Package nvarchar(170) = 'AnotherSsisPackage.dtsx'
  91. DECLARE @Step2_Command nvarchar(1700) =
  92. '/FILE "\\shared\etl\MyJob\AnotherSsisPackage.dtsx.dtsx"' +
  93. ' /CHECKPOINTING OFF' +
  94. ' /SET "\Package.Variables[User::EtlServer].Value";"localhost"' +
  95. ' /SET "\Package.Variables[User::EtlDatabase].Value";"etl_config"' +
  96. ' /SET "\Package.Variables[User::SsisLogServer].Value";"localhost"' +
  97. ' /SET "\Package.Variables[User::SsisLogDatabase].Value";"etl_config"' +
  98. ' /REPORTING E'
  99. EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=@Step2_Name,
  100. @step_id=2,
  101. @on_success_action=3,
  102. @on_fail_action=2,
  103. @subsystem=N'SSIS',
  104. @command=@Step2_Command
  105. IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
  106. -- ---------------------------------------------
  107. -- Job Schedule
  108. -- ---------------------------------------------
  109. -- ----------------------------------------------
  110. -- Job Alert
  111. -- ----------------------------------------------
  112. -- ---------------------------------------------
  113. -- Set start step
  114. -- ---------------------------------------------
  115. EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
  116. IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
  117. -- ---------------------------------------------
  118. -- Set server
  119. -- ---------------------------------------------
  120. EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = '(local)'
  121. IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
  122. PRINT 'Done!'
  123. COMMIT TRANSACTION
  124. GOTO EndSave
  125. QuitWithRollback:
  126. IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
  127. EndSave:
  128. GO

Summary

I hope that this set of suggestions has thrown a new light on what F# can be used for.

In my opinion, the combination of concise syntax, lightweight scripting (no binaries) and SQL type providers makes
F# incredibly useful for database related tasks.

Please leave a comment and let me know what you think.