Category Archives

46 Articles

Passing command line settings to SQL Server Integration Services (SSIS) packages using dtexec on Linux

by Jeroen ter Heerdt 0 Comments

This is my first blog post since I moved to Redmond. A lot of time has passed, sorry for that! I kept busy filling out forms, forms and some more forms. After that came more forms. I dream about forms, I breath forms. I am a form. Wait, let’s stop there.

Recently I had to figure out how to pass in settings to an SQL Server Integration Services (SSIS) package when calling it using dtexec from bash on Linux.

The dtexec utility is utility to execute a SSIS packages. The name goes back to the time that SSIS was called Data Transformation Services (DTS) – that is also the reason why SSIS packages carry the .dtsx extension. The dtexec utility is available for both Windows and Linux.

One of the interesting things you can do with dtexec is passing in values to variables (using the /Par(ameter) option or even change a connection string at runtime (using the /Conn[ection]). I had to do the latter. Here is where the fun started. The /Par and /Conn options (and maybe others) expect something like this: [name];[value]. For example:

Notice the ‘;’ between the name of the source connection and the connection string. This is all great, unless you are trying to call dtexec from the bash command line in Linux, because ; actually means something in bash: when bash sees ; it thinks the current command has ended and the next instruction follows. That is not what we want here.

I had to escape a number of items to get this to work correct in bash:

See what I did there? I had to escape the first ; (between ‘MasterSQL’ and ‘Data Source’) as well as escape the quotes that surround the data source definition itself and have it double quoted.

If you need to pass in a value to a string parameter you need to follow the same logic:

For a string variable you can pass in the value like this:

Hope this helps!

Farewell Tour

by Jeroen ter Heerdt 0 Comments

DUTCH DATA DUDE
FAREWELL TOUR


A CHANGE IS GONNA COME
Na bijna 10 jaar waag ik de grote stap. Ik verhuis naar Amerika en ga voor Microsoft in Seattle aan de slag. Daarmee komt er een einde aan mijn werk als Dutch Data Dude in Nederland (en daarbuiten). Werk dat ik met veel passie heb gedaan en heb kunnen doen dankzij en met jullie. Om dit deel gepast af te sluiten doe ik een FAREWELL TOUR: ik klim nog een laatste keer op het podium. Bij jullie, voor jullie. Want jullie hebben mij in staat gesteld te worden wie ik wilde zijn: het geluid van Microsoft rondom data en Artificial Intelligence. Dutch Data Dude signs off.

DE TOUR

Tijdens de FAREWELL TOUR geef ik in korte tijd een flink aantal presentaties.
Dit organiseer ik samen met partners in October / November 2018. Het onderwerp van de presentaties is Business Intelligence, Big Data, Machine Learning of Artificial Intelligence. Soms staat mijn presentatie op zich, soms is het onderdeel van een groter geheel. Alles in samenspraak met de organiserende partner. Ik hoop je te mogen begroeten tijdens de FAREWELL TOUR.
DE LOCATIES

Hieronder staan de locaties die ik aandoe tijdens mijn FAREWELL TOUR. Voor publieke evenementen kun je je direct inschrijven via de aangegeven link.

Datum Tijd Plaats Organisatie Soort evenement Inschrijven
17 October 10:00 Bussum CraniumConnect Voor leden Website
17 October 17:00 Den Haag TopBI Intern n.v.t.
18 October 17:00 Amsterdam QNH Intern n.v.t.
23 October 16:00 Amsterdam Datafied Publiek Inschrijven
25 October 18:00 Rotterdam Motion10 Publiek Inschrijven
29 October 16:30 Eindhoven Valid Publiek Inschrijven
30 October 13:30 Veenendaal HSO Op uitnodiging Website
30 October 16:00 Huizen Sigma Data Consulting Intern n.v.t.
31 October 17:00 Vianen Rubicon Publiek Inschrijven
2 November 09:00 Amsterdam Breinwave Op uitnodiging n.v.t.
6 November 13:30 Veenendaal Hillstar Publiek Inschrijven
7 November 19:00 Veenendaal Infosupport Publiek (beperkt aantal plaatsen) Inschrijven
8 November 14:30 Hoofddorp Macaw Publiek Inschrijven
12 November 13:30 Utrecht Axians Intern n.v.t.
13 November 17:30 Utrecht Caesar Experts Publiek Inschrijven
14 November 10:00 Bussum CraniumConnect Voor leden Website
14 November 18:00 Vianen Sogeti Intern n.v.t.
15 November 15:00 Amsterdam Ireckonu Op uitnodiging n.v.t.
29 November 17:00 Den Bosch Power BI Gebruikersgroep Publiek Inschrijven

SQL Server 2017: It’s here!

by Jeroen ter Heerdt 0 Comments

Anxiously you have waited for this day. The day you could finally get your hands on the latest version of the best database platform available. Today is that day. SQL Server 2017 is available and it runs on Windows, Linux and Docker. Find more info here: https://www.microsoft.com/en-us/sql-server/sql-server-2017. What changed? So many things it is hard to mention them all. Have a look at the Ignite sessions to get a feel for what happened.

 

SQL Server news from connect start to resonate

by Harry Tolsma 0 Comments

Hi all,

A month ago at the Connect event in New York Microsoft did some anouncements on SQL Server vNext and Azure Data Services.
It’s good to see the traction it’s getting!

I work with a lot of partners on implementing Microsoft stuff and the partners are very keen in getting the latest info on new capabilities.
Also they enlist in new online and in classroom courses on the new stuff. So let’s take a quick look:

  • Azure Data Lake store and analytics went live in Novmeber, This is a no limits data lake with on-demand analytics that instantly scales to your needs. With petabyte size files and trillions of objects and parallel processing!
  • R Server for Azure HD Insight for Largest R compatible parallel analytics ML library with terabyte-scale machine learning!
  • SQL Server on Linux public preview is released. Check in out on a linux box!
  • Moving selected Enterprise Edition capabilities to SQL Server Standard like in memory and Polybase.

And last but not least: connectivity to all top programming languages!

 

Power BI Refresh scenarios against data storage solutions explained

by Jeroen ter Heerdt 0 Comments

A recurring theme with customers and partners is Power BI data refresh. More specifically, there is some confusion on what refresh scenario requires a Pro version and what can be done with the Free version of Power BI. I made the diagram below to help explain this. It shows the refresh scenarios against data storage solutions, such as SQL Azure, SQL in a virtual machine in Azure or SQL server on premises. I used these as examples, there are other options as well. I think the overall time carries over to other data storage solutions. The diagram shows the refresh that can be done using a Power BI Free account as orange and the refresh scenarios that need Power BI Pro as green lines. As shown in the diagram, if  you want to refresh against on-premises sources or a database running in a VM in Azure you will need a gateway and Power BI Pro. This applies not only to the creator of the report and schedule but also to every consumer. If you use PAAS solutions for data storage in Azure such as SQL Azure, it becomes a bit more difficult and it is really dependent on the type of refresh required. If you need a refresh cycle higher than once a day (either max 8 times per 24 hours or live) you will need Power BI Pro. If you just want to refresh against such as SQL Azure and once a day is enough you can do that using Power BI Free. Again, the license requirement carries over from author to viewer; if the author of the report requires Pro, then the viewers also need Pro.

Power BI Refresh scenarios against data storage solutions

Hope this helps. If you have any questions or feedback, please comment below!

SQL Server op Linux? Ja dus!

by Harry Tolsma 0 Comments

Als iemand een paar jaar terug tegen mij had gezegd dat Microsoft Multi platform zou gaan en dat ook SQL Server op Linux zou uitkomen, dan had ik hem voor gek verklaard. Echter sinds de komst van Satya Nadella als CEO en mensen als Scott Guthrie (CVP voor Cloud & Enterprise) waait er een frisse wind door Microsoft!

En als Microsoft employee voelde ik natuurlijk al wat aankomen, maar het is toch altijd weer top om het nieuws dan nu in de buitenwereld te zien landen: Gister heeft Microsoft de Linux versie van SQL Server geannonceerd in preview. Dit betreft voorlopig alleen het RDBMS en niet de overige SQL tools als SSRS en SSAS, maar een mooie start om nu ook op Linux de (volgens Gartner) de leading database te kunnen runnen!

Ik zou het leuk vinden als jullie na het testen van deze versie op het blog zouden willen reageren met de eerste indruk van de performance van deze preview versie.

De blogpost van Scott over de annoncering: http://blogs.microsoft.com/blog/2016/03/07/announcing-sql-server-on-linux/

SQL groeten,

Harry

Automatically building a Microsoft BI machine using PowerShell – Final post (post #14)

by Jeroen ter Heerdt 0 Comments

This post is #14 in the series to automatically build a Microsoft BI machine using PowerShell – see the start of series.

In this series:

Start of series – introduction and layout of subjects
Post #2 – Preparation: install files using Azure disk
Post #3 – Preparation: install files using Azure File Service
Post #4 –Preparation: logging infrastructure
Post #5 – Master script
Post #6 – Disabling Internet Explorer Enhanced Security Configuration
Post #7 – Active Directory setup
Post #8 – Configuring Password policy
Post #9 – Installing System Center Endpoint Protection
Post #10 – Installing SQL Server
Post #11 – Installing SharePoint Server
Post #12 – Installing PowerPivot for SharePoint
Post #13 – Configuring PowerPivot for SharePoint

Wow. This has been a long and wild ride. But you and I made it together. We now have the full recipe to automatically configure a Microsoft BI demo machine with PowerShell. Of course there is more to be done, such as configuring other Service Accounts and deploying demo content; this script however saves me a lot of time every time I need to stand up a new demo machine.

You can download the script on Github. Please note (again) that the code is provided as-is and you should use it at your own risk. It is probably still buggy but should give you a good starting point to adapt it to your needs.

I enjoyed the ride with you; hope I made your life a bit easier of the course of this series. Enjoy!

Automatically building a Microsoft BI machine using PowerShell – Configuring PowerPivot (post #13)

by Jeroen ter Heerdt 0 Comments

This post is #13 in the series to automatically build a Microsoft BI machine using PowerShell – see the start of series.

In this series so far:

Start of series – introduction and layout of subjects
Post #2 – Preparation: install files using Azure disk
Post #3 – Preparation: install files using Azure File Service
Post #4 –Preparation: logging infrastructure
Post #5 – Master script
Post #6 – Disabling Internet Explorer Enhanced Security Configuration
Post #7 – Active Directory setup
Post #8 – Configuring Password policy
Post #9 – Installing System Center Endpoint Protection
Post #10 – Installing SQL Server
Post #11 – Installing SharePoint Server
Post #12 – Installing PowerPivot for SharePoint

Now that PowerPivot for SharePoint has been installed, we need to configure it. I split the configuration into two parts since we need a reboot in between and used MSDN for reference: http://msdn.microsoft.com/en-us/library/hh230903.aspx.

Step A: configuring SharePoint and deploying PowerPivot features

In Post #11 we talked about installing SharePoint, but the actual SharePoint provisioning was not done then. We will do it here in one go with installing PowerPivot features.

 

Step B: updating farm credentials and starting service applications

After the PowerPivot features have been deployed we need to configure Service Applications to get PowerPivot to work.

 

Now we have seen all the steps required to build a Microsoft BI demo machine! The next post will serve as a wrap up and present a download for the full script.

Automatically building a Microsoft BI machine using PowerShell – Installing PowerPivot for SharePoint (post #12)

by Jeroen ter Heerdt 1 Comment

This post is #12 in the series to automatically build a Microsoft BI machine using PowerShell – see the start of series.

In this series so far:

Start of series – introduction and layout of subjects
Post #2 – Preparation: install files using Azure disk
Post #3 – Preparation: install files using Azure File Service
Post #4 –Preparation: logging infrastructure
Post #5 – Master script
Post #6 – Disabling Internet Explorer Enhanced Security Configuration
Post #7 – Active Directory setup
Post #8 – Configuring Password policy
Post #9 – Installing System Center Endpoint Protection
Post #10 – Installing SQL Server
Post #11 – Installing SharePoint Server

Ok, now that both SQL Server and SharePoint Server are installed, we just need to set up PowerPivot for SharePoint and configure it. Easy huh? Well, it turns out it is pretty difficult to get it right. Installation is not difficult (this post) but the configuration is harder (the next post). Here is how to install PowerPivot. I used MSDN for the info: http://msdn.microsoft.com/en-us/library/ee210645.aspx.

Installing PowerPivot involves mounting the SQL Server Installation Media and calling the setup with the right parameters.

Next time: configuring PowerPivot.

Automatically building a Microsoft BI machine using PowerShell – Installing SharePoint (post #11)

by Jeroen ter Heerdt 0 Comments

This post is #11 in the series to automatically build a Microsoft BI machine using PowerShell – see the start of series.

In this series so far:

Start of series – introduction and layout of subjects
Post #2 – Preparation: install files using Azure disk
Post #3 – Preparation: install files using Azure File Service
Post #4 –Preparation: logging infrastructure
Post #5 – Master script
Post #6 – Disabling Internet Explorer Enhanced Security Configuration
Post #7 – Active Directory setup
Post #8 – Configuring Password policy
Post #9 – Installing System Center Endpoint Protection
Post #10 – Installing SQL Server

Wow, so the last post was pretty intense, wasn’t it? I think we are ready for the next one: installing SharePoint. To build this script I used the following sources: http://social.technet.microsoft.com/wiki/contents/articles/14582.sharepoint-2013-install-prerequisites-offline-or-manually-on-windows-server-2012-a-comprehensive-guide.aspx#Installing_the_Roles_and_Features_for_SharePoint_2013_on_Windows_Server_2012_Offline_with_PowerShell and http://blogs.msdn.com/b/uksharepoint/archive/2013/03/18/scripted-installation-of-sharepoint-2013-and-office-web-apps-server-from-the-field-part-2.aspx.

Since this is again a quite lengthly script we will split it up in steps.

Step A: enabling IIS and other features

This step enables a whole load of features on Windows that are required by SharePoint, including IIS. If a restart is required, the script will reboot after the setup of the features. Some times your machine might reboot more than once to complete the setup of all these features.

 

Step B: Installing SharePoint Prerequisites

SharePoint itself has a number of prerequisites; in this still we will install them all.

 

Step C: Installing SharePoint

SharePoint is installed in this step from the ISO that the script mounts.

 

Step D: Cleaning up

This step simply unmounts the SharePoint installation media.

 

Pff, are we done yet? No! Next up: Installing PowerPivot for SharePoint.

%d bloggers like this: