Agile Data Warehouse – MetaShare

I haven’t had any time to work on MetaShare lately, but I know that someone has made it work with Katmai already.  I don’t think that the code was checked in, but if I get a chance, I’ll try and make the changes so there is a switch to allow operation with Katmai.

Also – I’ve had a couple of requests to perform a demo of MetaShare.  I’d be happy to do that.  Let me look into ways to do that and I’ll post somethig about a time and method.  I know that some folks are not necessarily in the US, so maybe we can figure out a way to do the audio over the internet as well as desktop sharing.  If anyone has suggestions, let me know for both audio and desktop sharing.  If there isn’t anyone from outside the US, then we can use something like http://www.freeconferencecall.com/ - but I’m still interested if anyone knows of a free desktop sharing solution we could use.

We’d cover breifly how I set up the demo environment with Microsoft Data Warehouse Toolkit sample data structure and SSIS packages written by Warren and Joy, how to extract metadata out of the MDWT_AdventureWorksDW database as well as the SSIS package store and publish this information as Wiki using FlexWiki.  We’ll also talk about how powerful it is to publish this metadata as wiki instead of something like a dead text document.

Maybe we could do something week after next – the week of the 10th?  Leave me a comment here if you’d be interested in attending.  We can also do multiple sessions if we can’t agree on a time.  I think it will take from a half hour to an hour to run through the whole thing.

Stay tuned.

Mark

Fragmentation Stats on an Index (Part 2)

In my last post I was discussing how and why clicking on the Fragmentation tab on the properties page of an index could take a while to return results.  I identified it as a DBCC since sp_who2 as well as being identified as a DBCC command by querying

SELECT * FROM sys.dm_exec_requests
CROSS APPLY sys.dm_exec_sql_text(sql_handle)

I guess the point I was trying to make was that index fragmentation stats don’t seem to be cached anywhere (why would they be since they are changing all the time) and must be calculated each time you click on the Fragmentation tab – or perform DBCC SHOWCONTIG – or query db_dm_physical_index_stats.  This means that on deep tables, these queries can take some time.  I’m not working with an amazing disk subsystem and the server I’m working on only has two processors.  So for me – for sure – they take a while.

If you don’t mind waiting for SSMS to finish the query to see the results, that’s great.  I had accidentally clicked on it and needed SSMS to release the thread so that that SSMS would operate properly.  I’m hoping that I was showing how to deal with SSMS hanging on this page when your data set is large.

Thanks to Simon Worth for pointing this out.  He is completely correct.  DBCC SHOWCONTIG has been retired for using db_dm_physical_index_stats instead.  SSMS is making a SELECT to db_dm_physical_index_stats and not really calling DBCC SHOWCONTIG.  I wouldn’t be suprised if under the covers that both DBCC SHOWCONTIG and a call to db_dm_physical_index_stats are doing the exact same operations to figure out what the fragmentation stats are.  Microsoft has just put a new front end on to DBCC SHOWCONTIG so that it looks more like a regular T-SQL SELECT statement.

Thanks, Simon.

Fragmentation Stats on an Index

I’m working with very large data sets and I accidentally clicked on the “Fragmentation” tab on the properties page for the clustered index of a table.  Management Studio did not handle this very gracefully.  Everything locked up and Management Studio becomes un-usable.

What Managment Studio is doing is issuing a DBCC on the index to see what the fragmentation stats are.  What makes this bad is that this table has 100,000,000 rows in it.  Needless to say, this takes a while to complete.

I decided that I wasn’t willing to wait for the DBCC to finish so I used a KILL statement to end the process.  Everything came back to life and I was able to dismiss the index properties dialog.

I wonder if it would be nice for SQL Server to have some idea if the DBCC will take a while and if it will, maybe it can be treaded off so that it doesn’t hang SSMS.  Just a thought.