This project has moved and is read-only. For the latest updates, please go here.
2
Vote

Add support for process partitioning (and ProcessAdd)

description

I would improve the process support and in particular I would add the support for ProcessAdd with partitions, as I described in this article:
http://www.sqlbi.com/articles/incremental-processing-in-tabular-using-process-add/

comments

JuanPabloJofre wrote Aug 20, 2012 at 6:14 PM

Thanks Marco!!
Would you like to contribute to the project and include your suggestion to the code base?
I would really appreciate to have members from the community contributing to the sample.
With my best regards,

JuanPablo Jofré

marcorusso wrote Aug 21, 2012 at 6:53 AM

Hi JuanPablo,

yesterday I had to implement a C# program to automate incremental processing and initially I tried to extend Tabular AMO (so it would have been possible to contribute to the code). However, at the end I implemented it in a way that is independent from Tabular AMO. The problem is that I didn't had time to figure out to include in the Tabular AMO object model the following features:
  • query binding for specific process operations
  • transactions for including more operations within the same database
After having written it, I realized that the only really missing part is the QueryBinding support. A simple way could be adding a string to the Process function for Partition and Table, using the existing DataSourceView for connection. However, I think that a more general approach is allowing the change of the connection for the specific process operation (also for ProcessAdd it might be useful to connect to a different data source).
Do you think is ok to put an optional string parameter that should correspond to an existing DataSourceID in the Process functions?
If yes, maybe we can just add two optional parameters (query and dataSourceId) to table and partition process functions.

I still am not familiar with the overall design of the Tabular AMO library and I'd like to get your feedback before trying to implement this feature in existing code.
Thanks,

Marco