Showing posts with label index. Show all posts
Showing posts with label index. Show all posts

Monday, March 26, 2012

Relation between reads and duration

I have a query that performs strangely under different conditions as
follows:
When ran normally, it uses a clustered index scan. From the Profiler and
the IO statistics, this makes ~250,000 reads. This takes about 4 minutes to
run.
When ran with an index hint, it uses a clusted index seek and makes a
bookmark lookup of about 180,000 rows. Reads are ~1,000,000. However, this
takes only 1 minute to run.
Why is it like that? I have tried defragging the clustered index, but to no
avail. Running another query that also makes a clustered index scan makes
~250,000 reads, and takes ~ 4 minutes too.
What else can I check? Why does a query that makes less reads takes longer
to run than another that makes 4 times as many reads? I would like to avoid
having to force the hint, if I can help it.
Thx in advance.
Regards
Ray MondRay,
Do the 250K reads from the clustered index involve wide rows, so they
would access many data pages (maybe even 250,000)? With the hint, maybe
most of the million reads are from a relatively small number of pages in
the nonclustered index, and therefore from memory, not disk.
There are lots of factors involved, and sometimes there just isn't
enough information for the optimizer to choose the best plan. If you
provide some more information about your query (such as create table
statements, indexes, the actual queries and maybe even the good and bad
plans you're seeing), maybe we can give more specific help.
SK
Ray Mond wrote:

>I have a query that performs strangely under different conditions as
>follows:
>When ran normally, it uses a clustered index scan. From the Profiler and
>the IO statistics, this makes ~250,000 reads. This takes about 4 minutes t
o
>run.
>When ran with an index hint, it uses a clusted index seek and makes a
>bookmark lookup of about 180,000 rows. Reads are ~1,000,000. However, thi
s
>takes only 1 minute to run.
>Why is it like that? I have tried defragging the clustered index, but to n
o
>avail. Running another query that also makes a clustered index scan makes
>~250,000 reads, and takes ~ 4 minutes too.
>What else can I check? Why does a query that makes less reads takes longer
>to run than another that makes 4 times as many reads? I would like to avoi
d
>having to force the hint, if I can help it.
>Thx in advance.
>Regards
>Ray Mond
>
>|||Steve,
Average row size is 1,248 bytes. You are probably right, in that the
execution plan using the non-clustered index is reading data from pages
already in memory, because the data rows requested by the query are bunched
up together and not randomly distributed. Is there a way to know the actual
number of unique pages actually looked up by a query, short of peeking into
the contents of each page?
Thanks.
Regards
Ray Mond
"Steve Kass" <skass@.drew.edu> wrote in message
news:OBrtou$BEHA.2620@.TK2MSFTNGP12.phx.gbl...
> Ray,
> Do the 250K reads from the clustered index involve wide rows, so they
> would access many data pages (maybe even 250,000)? With the hint, maybe
> most of the million reads are from a relatively small number of pages in
> the nonclustered index, and therefore from memory, not disk.
> There are lots of factors involved, and sometimes there just isn't
> enough information for the optimizer to choose the best plan. If you
> provide some more information about your query (such as create table
> statements, indexes, the actual queries and maybe even the good and bad
> plans you're seeing), maybe we can give more specific help.
> SK
> Ray Mond wrote:
>
to
this
no
makes
longer
avoid
>|||I don't know of a way, though maybe there's something in profiler that
I've never seen. I don't recall the details now, but I think the beta 1
release of SQL Server 2005 had some extra show statistics io column -
maybe the situation will improve in future versions of SQL Server.
SK
Ray Mond wrote:

>Steve,
>Average row size is 1,248 bytes. You are probably right, in that the
>execution plan using the non-clustered index is reading data from pages
>already in memory, because the data rows requested by the query are bunched
>up together and not randomly distributed. Is there a way to know the actua
l
>number of unique pages actually looked up by a query, short of peeking into
>the contents of each page?
>Thanks.
>
>|||Steve,
Thx. I'm using the SET STATISTICS IO output to get a ballpark figure of the
number of unique pages read.
Regards
Ray Mond
"Steve Kass" <skass@.drew.edu> wrote in message
news:%23RMkgJFCEHA.3064@.tk2msftngp13.phx.gbl...
> I don't know of a way, though maybe there's something in profiler that
> I've never seen. I don't recall the details now, but I think the beta 1
> release of SQL Server 2005 had some extra show statistics io column -
> maybe the situation will improve in future versions of SQL Server.
> SK
> Ray Mond wrote:
>
bunched
actual
into
>sql

Monday, March 12, 2012

Re-indexing table causes Windows to re-boot !

I have a DB with multiple tables and when I attempt to index one of the
tables the whole Windows servers crashes and reboots. My understanding is
that SQLServer should run as an isolated application and so should not be
able to affect the operating system.
This server has been fine for over a year now but has just started to
exhibit this problem (no recent changes or updates to the server, apart from
regular anti-virus updates, that could account for this behaviour).
The problem in more detail:
Database - 1577MB data (518 MB free). Set to automatically grow at 10% to
an un-restricted limit (data and log)
Table in question - 900808 KB data, ~7000000 rows of data
Indexes - there are three "user" indexes that I create. One of them is a
clustered index. When any of these indexes is built the server crashes.
Exhaustive DBCC tests imply that the DB and table are in perfect health.
Nothing in the event log and no memory dumps.
SQLServer - SQLServer 2000 sp3a
Windows - Win2000 sp3, IBM XSERIES_335 with 3 GHz xeon processor and oodles
of memory.
User - at wits end
Any advice would be most welcome
Thanks
GriffGriff,
You are correct. You should focus on the operating system. Most likely SQL
Server is just taxing the IO or Processor hard and there is either a
hardware or driver issue. In the event log is it an unexpected reboot or is
there a stop error listed?
"Griff" <Howling@.The.Moon> wrote in message
news:%234bILC2vFHA.2556@.TK2MSFTNGP15.phx.gbl...
>I have a DB with multiple tables and when I attempt to index one of the
>tables the whole Windows servers crashes and reboots. My understanding is
>that SQLServer should run as an isolated application and so should not be
>able to affect the operating system.
> This server has been fine for over a year now but has just started to
> exhibit this problem (no recent changes or updates to the server, apart
> from regular anti-virus updates, that could account for this behaviour).
> The problem in more detail:
> Database - 1577MB data (518 MB free). Set to automatically grow at 10% to
> an un-restricted limit (data and log)
> Table in question - 900808 KB data, ~7000000 rows of data
> Indexes - there are three "user" indexes that I create. One of them is a
> clustered index. When any of these indexes is built the server crashes.
> Exhaustive DBCC tests imply that the DB and table are in perfect health.
> Nothing in the event log and no memory dumps.
> SQLServer - SQLServer 2000 sp3a
> Windows - Win2000 sp3, IBM XSERIES_335 with 3 GHz xeon processor and
> oodles of memory.
> User - at wits end
> Any advice would be most welcome
> Thanks
> Griff
>
>|||Having read the event logs (application, system, security & SQLServer
ones) and searched for a memory dmp file I get the impression that the
server crashed so fast that it simply didn't have the chance to log
anything. Metaphorically speaking, it had the rug pulled from under
it.
The event log simply lists the fact that it rebooted after an
unexpected system shutdown, but I can't find anything that implies what
could have caused the system shutdown.
Watching the task manager whilst the re-indexing is occuring (before it
dies), it shows no real sign of taxing the server's resources. Of
course now, running live without indexes, the server is as slow as a
lame dog, presumably because SQLServer is having to do lots of full
table scans.|||Oddly enough I've seen this scenerio with two of our 32 way servers. We and
the hardware vendor were able to reproduce the system fault but the vendor
never could tell us what was wrong or how to fix it. Basically when the
server approached 100% CPU (doing masive qeries) the system started geting
dirty memory transfers between pods. It all started after we upgraded our
SQL Server 2000 build. We switched to 64 bit HP servers and immediately
stopped experiencing the issue. Assuming a good power supply, in my opinion
any unexpected reboot on a Windows server is one of three things.
From most likely to least likely:
1. A hardware issue
2. A poorly written driver in kernal mode
3. Windows not trapping the error.
SQL Server may be pushing the system in a new way or to 100% but that would
be a performance issue not an availability issue.
Good luck.
"Griff" <griffithsj_520@.hotmail.com> wrote in message
news:1127390278.062254.254300@.g49g2000cwa.googlegroups.com...
> Having read the event logs (application, system, security & SQLServer
> ones) and searched for a memory dmp file I get the impression that the
> server crashed so fast that it simply didn't have the chance to log
> anything. Metaphorically speaking, it had the rug pulled from under
> it.
> The event log simply lists the fact that it rebooted after an
> unexpected system shutdown, but I can't find anything that implies what
> could have caused the system shutdown.
> Watching the task manager whilst the re-indexing is occuring (before it
> dies), it shows no real sign of taxing the server's resources. Of
> course now, running live without indexes, the server is as slow as a
> lame dog, presumably because SQLServer is having to do lots of full
> table scans.
>|||Well, I can't say what the problem was, but we do appear to have fixed it.
Despite nothing "appearing" to have changed on the system (we manually apply
the Microsoft critical upgrades), this problem started occuring out of the
blue.
However, it got worse throughout the day - the server started rebooting
during "normal" use, approximately ever 30 minutes. As mentioned
previously, the CPU was more or less at 100% because we couldn't rebuild the
indexes, so I was wondering whether it could be overheating or some other
symptom of the hardware being over-stressed (though it's a good spec server
so should cope).
Anyhow, that evening we clutched at straws and applied Win2000 sp4 (had been
sp3) and, would you believe it, it fixed the problem and a good nights sleep
was had by all.|||Griff wrote:
> Well, I can't say what the problem was, but we do appear to have
> fixed it.
> Despite nothing "appearing" to have changed on the system (we
> manually apply the Microsoft critical upgrades), this problem started
> occuring out of the blue.
> However, it got worse throughout the day - the server started
> rebooting during "normal" use, approximately ever 30 minutes. As
> mentioned previously, the CPU was more or less at 100% because we
> couldn't rebuild the indexes, so I was wondering whether it could be
> overheating or some other symptom of the hardware being over-stressed
> (though it's a good spec server so should cope).
> Anyhow, that evening we clutched at straws and applied Win2000 sp4
> (had been sp3) and, would you believe it, it fixed the problem and a
> good nights sleep was had by all.
Probably someone was "joking": there are some holes in some windows
versions which allow a host to be restarted / switched off or similar via
network (these are usually closed by sp4 IIRC). So it could even be
manual intervention that caused this...
Kind regards
robert|||Hi Robert
I believe that we can almost rule that one out. The server is behind a
firewall that only allows HTTP in from unknown IP addresses, everything else
is effectively blocked. The only person who can connect to it in any other
way is me.
But hey, we know that the only way to make a computer completely safe is to
switch it off.

Reindexing 27GB table

On one of the database, I support has 16957722 records in a table and it's
clustered index size is 27606.30.
I used to run the Reindex job created on weekly Sunday starting at 11:00 PM.
It takes more than 7 hours.
Since it is a 24x7 server, when Reindexing job runs it causes blocks so I am
cancelling the job.
In this situation what would you suggest to reindex the table of
such big one ( 27 GB ) ?
Appreciate your advise.
--
Thanks
S SIVAPRASADFirst you need to determine whether you benefit from reindexing. See
[url]http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx.[/url
]
To check frag level use DBCC SHOWCONTIG (2000) or sys.dm_db_index_physical_s
tats (2005).
In 2000, you have little options of you want to do this "online", your optio
n is DBCC INDEXDEFRAG.
In 2005, you can use either ALTER INDEX with REORGANIZE (same as INDEXDEFRAG
), or use ALTER INDEX
with REBUILD, ONLINE (same as DBREINDEX, except for ONLINE which is new opti
on).
All have pro and cons, one thing to consider is the usage of transaction log
space (which depends on
what type of command you run and can also be influenced by recovery model).
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"S Siva Prasad [SIVA]" <SSivaPrasadSIVA@.discussions.microsoft.com> wrote
in message
news:2484E248-2FCD-47F8-BACB-0AB9E71DEAC3@.microsoft.com...
> On one of the database, I support has 16957722 records in a table and it's
> clustered index size is 27606.30.
> I used to run the Reindex job created on weekly Sunday starting at 11:00 P
M.
> It takes more than 7 hours.
> Since it is a 24x7 server, when Reindexing job runs it causes blocks so I
am
> cancelling the job.
> In this situation what would you suggest to reindex the table of
> such big one ( 27 GB ) ?
> Appreciate your advise.
> --
> Thanks
> S SIVAPRASAD|||It is on SQL Server 2000 Enterprise Edition with SP4 (8.00.2040) .
Database is on Full recovery model and it has enough space for Log file
growth.
DBCC SHOWCONITG() ran and rebuilding of the indexes is necessary now.
Let me know what would be the best optimal way of rebuilding the indexes.
The box is on 24x7 and I want to rebuild the index as quick as possible.
The clustered index sized around 27 GB.
--
Thanks
S SIVAPRASAD
"Tibor Karaszi" wrote:

> First you need to determine whether you benefit from reindexing. See
> [url]http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx.[/u
rl]
> To check frag level use DBCC SHOWCONTIG (2000) or sys.dm_db_index_physical
_stats (2005).
> In 2000, you have little options of you want to do this "online", your opt
ion is DBCC INDEXDEFRAG.
> In 2005, you can use either ALTER INDEX with REORGANIZE (same as INDEXDEFR
AG), or use ALTER INDEX
> with REBUILD, ONLINE (same as DBREINDEX, except for ONLINE which is new op
tion).
> All have pro and cons, one thing to consider is the usage of transaction l
og space (which depends on
> what type of command you run and can also be influenced by recovery model)
.
> --
> Tibor Karaszi, SQL Server MVP
> http://www.karaszi.com/sqlserver/default.asp
> http://www.solidqualitylearning.com/
>
> "S Siva Prasad [SIVA]" <SSivaPrasadSIVA@.discussions.microsoft.com> wro
te in message
> news:2484E248-2FCD-47F8-BACB-0AB9E71DEAC3@.microsoft.com...
>
>|||There's no "optimal way". There are different options, all with advantages a
nd disadvantages. Based
on your requirements, you can weigh the advantages to the disadvantages and
pick the one that suits
your particular need best. Make sure you understand the pros and cons with e
ach method and read the
whitepaper I posted a link to.
Having said that, if the box is 24*7, indexdefrag might be better. But it ca
n take longer time than
dbreindex (but it doesn't block nearly as much), it can produce more log rec
ords, and the
defragmentation might not be as perfect as dbreindex.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"S Siva Prasad [SIVA]" <SSivaPrasadSIVA@.discussions.microsoft.com> wrote
in message
news:8A93A9FA-E357-475D-BB40-8F9BE0A43CCF@.microsoft.com...[vbcol=seagreen]
> It is on SQL Server 2000 Enterprise Edition with SP4 (8.00.2040) .
> Database is on Full recovery model and it has enough space for Log file
> growth.
> DBCC SHOWCONITG() ran and rebuilding of the indexes is necessary now.
> Let me know what would be the best optimal way of rebuilding the indexes.
> The box is on 24x7 and I want to rebuild the index as quick as possible.
> The clustered index sized around 27 GB.
> --
> Thanks
> S SIVAPRASAD
>
> "Tibor Karaszi" wrote:
>

Reindexing 27GB table

On one of the database, I support has 16957722 records in a table and it's
clustered index size is 27606.30.
I used to run the Reindex job created on weekly Sunday starting at 11:00 PM.
It takes more than 7 hours.
Since it is a 24x7 server, when Reindexing job runs it causes blocks so I am
cancelling the job.
In this situation what would you suggest to reindex the table of
such big one ( 27 GB ) ?
Appreciate your advise.
--
Thanks
S SIVAPRASADFirst you need to determine whether you benefit from reindexing. See
http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx.
To check frag level use DBCC SHOWCONTIG (2000) or sys.dm_db_index_physical_stats (2005).
In 2000, you have little options of you want to do this "online", your option is DBCC INDEXDEFRAG.
In 2005, you can use either ALTER INDEX with REORGANIZE (same as INDEXDEFRAG), or use ALTER INDEX
with REBUILD, ONLINE (same as DBREINDEX, except for ONLINE which is new option).
All have pro and cons, one thing to consider is the usage of transaction log space (which depends on
what type of command you run and can also be influenced by recovery model).
--
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"S Siva Prasad [SIVA]" <SSivaPrasadSIVA@.discussions.microsoft.com> wrote in message
news:2484E248-2FCD-47F8-BACB-0AB9E71DEAC3@.microsoft.com...
> On one of the database, I support has 16957722 records in a table and it's
> clustered index size is 27606.30.
> I used to run the Reindex job created on weekly Sunday starting at 11:00 PM.
> It takes more than 7 hours.
> Since it is a 24x7 server, when Reindexing job runs it causes blocks so I am
> cancelling the job.
> In this situation what would you suggest to reindex the table of
> such big one ( 27 GB ) ?
> Appreciate your advise.
> --
> Thanks
> S SIVAPRASAD|||It is on SQL Server 2000 Enterprise Edition with SP4 (8.00.2040) .
Database is on Full recovery model and it has enough space for Log file
growth.
DBCC SHOWCONITG() ran and rebuilding of the indexes is necessary now.
Let me know what would be the best optimal way of rebuilding the indexes.
The box is on 24x7 and I want to rebuild the index as quick as possible.
The clustered index sized around 27 GB.
--
Thanks
S SIVAPRASAD
"Tibor Karaszi" wrote:
> First you need to determine whether you benefit from reindexing. See
> http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx.
> To check frag level use DBCC SHOWCONTIG (2000) or sys.dm_db_index_physical_stats (2005).
> In 2000, you have little options of you want to do this "online", your option is DBCC INDEXDEFRAG.
> In 2005, you can use either ALTER INDEX with REORGANIZE (same as INDEXDEFRAG), or use ALTER INDEX
> with REBUILD, ONLINE (same as DBREINDEX, except for ONLINE which is new option).
> All have pro and cons, one thing to consider is the usage of transaction log space (which depends on
> what type of command you run and can also be influenced by recovery model).
> --
> Tibor Karaszi, SQL Server MVP
> http://www.karaszi.com/sqlserver/default.asp
> http://www.solidqualitylearning.com/
>
> "S Siva Prasad [SIVA]" <SSivaPrasadSIVA@.discussions.microsoft.com> wrote in message
> news:2484E248-2FCD-47F8-BACB-0AB9E71DEAC3@.microsoft.com...
> > On one of the database, I support has 16957722 records in a table and it's
> > clustered index size is 27606.30.
> >
> > I used to run the Reindex job created on weekly Sunday starting at 11:00 PM.
> > It takes more than 7 hours.
> >
> > Since it is a 24x7 server, when Reindexing job runs it causes blocks so I am
> > cancelling the job.
> >
> > In this situation what would you suggest to reindex the table of
> > such big one ( 27 GB ) ?
> >
> > Appreciate your advise.
> > --
> > Thanks
> > S SIVAPRASAD
>
>|||There's no "optimal way". There are different options, all with advantages and disadvantages. Based
on your requirements, you can weigh the advantages to the disadvantages and pick the one that suits
your particular need best. Make sure you understand the pros and cons with each method and read the
whitepaper I posted a link to.
Having said that, if the box is 24*7, indexdefrag might be better. But it can take longer time than
dbreindex (but it doesn't block nearly as much), it can produce more log records, and the
defragmentation might not be as perfect as dbreindex.
--
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"S Siva Prasad [SIVA]" <SSivaPrasadSIVA@.discussions.microsoft.com> wrote in message
news:8A93A9FA-E357-475D-BB40-8F9BE0A43CCF@.microsoft.com...
> It is on SQL Server 2000 Enterprise Edition with SP4 (8.00.2040) .
> Database is on Full recovery model and it has enough space for Log file
> growth.
> DBCC SHOWCONITG() ran and rebuilding of the indexes is necessary now.
> Let me know what would be the best optimal way of rebuilding the indexes.
> The box is on 24x7 and I want to rebuild the index as quick as possible.
> The clustered index sized around 27 GB.
> --
> Thanks
> S SIVAPRASAD
>
> "Tibor Karaszi" wrote:
>> First you need to determine whether you benefit from reindexing. See
>> http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx.
>> To check frag level use DBCC SHOWCONTIG (2000) or sys.dm_db_index_physical_stats (2005).
>> In 2000, you have little options of you want to do this "online", your option is DBCC
>> INDEXDEFRAG.
>> In 2005, you can use either ALTER INDEX with REORGANIZE (same as INDEXDEFRAG), or use ALTER INDEX
>> with REBUILD, ONLINE (same as DBREINDEX, except for ONLINE which is new option).
>> All have pro and cons, one thing to consider is the usage of transaction log space (which depends
>> on
>> what type of command you run and can also be influenced by recovery model).
>> --
>> Tibor Karaszi, SQL Server MVP
>> http://www.karaszi.com/sqlserver/default.asp
>> http://www.solidqualitylearning.com/
>>
>> "S Siva Prasad [SIVA]" <SSivaPrasadSIVA@.discussions.microsoft.com> wrote in message
>> news:2484E248-2FCD-47F8-BACB-0AB9E71DEAC3@.microsoft.com...
>> > On one of the database, I support has 16957722 records in a table and it's
>> > clustered index size is 27606.30.
>> >
>> > I used to run the Reindex job created on weekly Sunday starting at 11:00 PM.
>> > It takes more than 7 hours.
>> >
>> > Since it is a 24x7 server, when Reindexing job runs it causes blocks so I am
>> > cancelling the job.
>> >
>> > In this situation what would you suggest to reindex the table of
>> > such big one ( 27 GB ) ?
>> >
>> > Appreciate your advise.
>> > --
>> > Thanks
>> > S SIVAPRASAD
>>

Friday, March 9, 2012

Regular SQL Index

What is Regular SQL Index? Where can I read more about the subject? Or how do can I use it in my tabels?not sure I have heard this one before UNLESS "Regular SQL Index" refers to a NON-Clustered index. Where did you hear about it?|||Originally posted by Paul Young
not sure I have heard this one before UNLESS "Regular SQL Index" refers to a NON-Clustered index. Where did you hear about it?

Here http://www.databasejournal.com/features/mssql/article.php/1438211#part_3|||The article was talking about Full-text search indexes as compared to indexes on a column or columns.