Search

OakieTags

Who's online

There are currently 0 users and 35 guests online.

Recent comments

Oakies Blog Aggregator

Fasthosts : Good Riddance…

It’s been a couple of weeks since I stopped hosting my website with Fasthosts. It’s too early to start singing the praises of the new hosting company, but so far so good.

As a parting shot, when I asked Fasthosts to cancel my hosting account I asked for a refund on the 8 months of outstanding service I had paid for. I got the answer back today and it was a definite no.

I’m not particularly surprised at this, but I thought as a good will gesture to a customer of 11 years they may do the decent thing and give me my money back. It’s not like I wanted to leave. I was forced to by their complete incompetence.

My advice to anyone considering using Fasthosts would be DONT!

Cheers

Tim…


Fasthosts : Good Riddance… was first posted on April 19, 2012 at 1:10 pm.
©2012 "The ORACLE-BASE Blog". Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement.




Repairman Jack : Gateways…

Gateways is the seventh book in the Repairman Jack series by F. Paul Wilson.

Jack’s dad has an “accident” and is left in a coma. Jack goes to visit him and tries to find out what happened. Another brush with “The Otherness” inevitably follows.

As I’ve come to expect, the backdrop to the story is quite sinister, but it moves too fast for you to dwell on that. It’s more like a full on action story and much less bleak than some of the previous stories. Maybe I’m just getting acclimatized… :)

Cheers

Tim…

PS. Having this cold has certainly allowed me to motor through some of these books.




Shrinking Tables to Aid Full Scans

{This blog is about shrinking tables where the High Water Mark is higher than it needs to be. Prompted by one of the comments, I wrote a follow-up post on finding the High Water Mark and tables that consist mostly of empty space, which would be candidates for shrinking.}

This blog is about one of those things I do as almost an autonomous “not thinking about it” performance housekeeping task, one which I have been meaning to mention for ages.

There can be quite a lot to gain by checking out full scans on “small” tables and seeing if it is as efficient as it can be. Sometimes it is, sometimes it is not. Often it is “good enough”. Occasionally it is awful.

Recently I was just casting an eye over the “top 20″ SQL on a system for any code generating a lot of consistent gets. I came across the below:

FRST_LOAD_TIME SQL_ID PRSE EXCS BUFFS DISCS RWS
-------------------- ------------- --------- ----------- -------------- ---------- ------------
CPU_MS ELAPSD_MS SORTS DIR_W OPT_COST
-------------- -------------- ---------- ---------- --------------
SQL_TEXT
----------------------------------------------------------------------------------------------------
2-03-14/18:00:10 bk9b5u6zyvy59 36,262 36,262 90,634,158 7 36,261
320,102.3 326,920.9 0 0 546
SELECT count(*) RUNNING_SESSIONS from SNP_SESSION WHERE SESS_STATUS = :1 and AGENT_NAME=:2

It caught my eye as it struck me as a lot of buffer gets for what I thought was quite a small table and it is executed quite often. In this case, a few thousand times a day. It takes 2500 buffer gets per count(*). All buffered, but the CPU has to chunk through it.

I check, it is a small table:

ABCD01> select count(*) from dev_ODI_XXXX.snp_session

COUNT(*)
----------
73

ABCD01> @tab_lst
Enter value for tab_name: snp_session

TABLE_NAME OWNER NUM_ROWS
------------------------------ ------------------------------ -------------
SNP_SESSION DEV_ODI_XXXX 49

-- and a quick check on those key columns

OWNER COLUMN_NAME NUM_DISTINCT N_NULLS LOW_V HI_V BKTS AVG_L
-------- -------------------- ------------ ---------- --------------- --------------- ---- -----
DEV_ODI_ SESS_STATUS 4 0 D W 4 2
DEV_ODI_ AGENT_NAME 4 6 AAA_ODI_AGENT BBB_ODI_AGENT 4 13

The table IS small, only 73 rows. It is also interesting that the stats are out by quite a percentage, but the table is still seen as small according to the stats. This indicates the table is quite volatile, but consistently small. Another thing to notice is that the number of distinct values for each column in the WHERE clause is only 4 and thus an index lookup would not be very specific. With such a small number of records in the table and not very specific index I “know” that an index lookup would not be of much benefit {BTW Everything that you “know” in this way – do check it in reality every now and again…}. I don’t look at adding an index, I look at why such a small table needs so many buffer gets.

So, why so many buffer gets?

First of all, I need to be careful. The original source of information I was looking as was V$SQL and that holds cumulative stats for all executions of the same SQL statement. Thus the buffer gets could have been very high a while ago (say, yesterday) and now are small, and the average is 2500 BGs per select but right now it is running fine. So I verify that the code is still doing a lot of work for such a small number of records:

set autotrace on
SELECT count(*) RUNNING_SESSIONS from dev_ODI_XXXX.SNP_SESSION 
WHERE SESS_STATUS = 'D' --:1 
and AGENT_NAME= 'DEV_ODI_AGENT'--:2

RUNNING_SESSIONS
----------------
1
Elapsed: 00:00:00.99

Execution Plan
----------------------------------------------------------
Plan hash value: 1837310352
------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 15 | 546 (1)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 15 | | |
|* 2 | TABLE ACCESS STORAGE FULL| SNP_SESSION | 1 | 15 | 546 (1)| 00:00:01 |
------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage("SESS_STATUS"='D' AND "AGENT_NAME"='DEV_ODI_AGENT')
filter("SESS_STATUS"='D' AND "AGENT_NAME"='DEV_ODI_AGENT')

Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
2013 consistent gets
0 physical reads

That’s still a lot of consistent gets to scan all of 75 rows and do a count (which is the sort aggregate step). A second to process and 2013 consistent gets to look at 75 rows in a skinny table – just trust me it is a skinny table. And the optimiser cost the full scan step at 546 “equivalent single IOs”.
{Oh, if anyone is wondering what the “table access storage full” and predicate step “storage(“SESS_STATUS”=’D’ AND “AGENT_NAME”=’DEV_ODI_AGENT’)” it’s because this system is an exadata box. Just ignore this for now, it is not relevant to this issue. I’ll blog about it *next week*}.

What is the problem?

Well, let’s see how big the table is.

ABCD01> @seg_dets
Enter value for seg_name: snp_session
Enter value for owner: dev_odi%
Any Key>

OWNER SEG_NAME SEG TS_NAME BYTES_K BLOCKS exts INI_K NXT_K
-------- --------------- --- -------- ---------- --------- ---- ------- -------
DEV_ODI_ SNP_SESSION TAB DEV_ODI_ 16,384 2,048 31 64 1024
REPO USER

The table is 2048 blocks in size, the consistent gets to look at the table is 2013. That’s close.

The problem is when you full scan a table, it scans up to the high water mark of that table, ie the highest block that Oracle knows there has been data in. As Oracle inserts new data into a table and extends it, it keeps a note of the highest block data has been put into, the High Water Mark (HWM). When you delete data from a table, Oracle marks individual blocks to be reused as they empty (drop below a certain percentage full). But Oracle does NOT check to see if many blocks are being fully emptied and it can reduce the high water mark. In fact, there could well be a few records in the first block of the table and then nothing until a few records in the last block that has been used in the table, the High Water Mark. Oracle just scans all blocks and counts relevant records up to the high water mark. This is actually a very long-standing performance tweek, you can allocate GBs of data to a table but Oracle knows the very highest block it has put data into for that segment and only scans up to the High Water Mark. Usually, the HWM is useful.

I happen to know that this table used to have several thousand records in it but is now used far less extensively and holds only a few records. They are constantly deleted and new ones created. I have seen {very regularly, at least once a year} some extreme situations where a table has had many millions of rows in it at one point, setting the HWM to several hundred thousand blocks into the table segment, but the table now only holds a few hundred records. But a Full Table Scan scans all the way up to the HWM, taking several minutes to access a few hundred records. Often that table is being inefficiently cleared with a “delete from table_x” statement and then re-populated.

What is the answer? Well, I could truncate the table and set the HWM to zero, but this table actually holds some real data. I can’t truncate it in this situation. I could MOVE the table, ie get Oracle to recreate the table and move the existing data into it. That would set the HWM of the new segment to block one and then raise it as it inserts the data neatly into the first few blocks. The third option, and the one I took, is to SHRINK the table. This basically tells oracle to shuffle the data down to the start of the table and reset the HWM. You can do this on the fly and I have done it on relatively active systems, but I must give the usual advice to test this yourself and you may have locking issues on a busy system hitting the table you shrink very hard. Also, you do need to enable row movement on the table for Shrink to work, as is demonstrated below:

ABCD01> alter table dev_ODI_XXXX.snp_session shrink space cascade;
alter table dev_ODI_XXXX.snp_session shrink space cascade
*
ERROR at line 1:
ORA-10636: ROW MOVEMENT is not enabled

ABCD01> alter table dev_ODI_XXXX.snp_session enable row movement;

Table altered.

ABCD01> alter table dev_ODI_XXXX.snp_session shrink space cascade;

Table altered.

Elapsed: 00:00:01.98

So, how does my little select perform now?

set autotrace on
SELECT count(*) RUNNING_SESSIONS from dev_ODI_XXXX.SNP_SESSION 
WHERE SESS_STATUS = 'D' --:1 
and AGENT_NAME= 'DEV_ODI_AGENT'--:2

RUNNING_SESSIONS
----------------
1
Elapsed: 00:00:00.04

Execution Plan
----------------------------------------------------------
Plan hash value: 1837310352
------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 15 | 546 (1)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 15 | | |
|* 2 | TABLE ACCESS STORAGE FULL| SNP_SESSION | 1 | 15 | 546 (1)| 00:00:01 |
------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage("SESS_STATUS"='D' AND "AGENT_NAME"='DEV_ODI_AGENT')
filter("SESS_STATUS"='D' AND "AGENT_NAME"='DEV_ODI_AGENT')

Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
8 consistent gets
0 physical reads

Fabulous. The run time is very short and the number of consistent gets is tiny. You might notice that the Explain Plan still thinks the cost of the full scan will be 546 (ie the equivalent of 546 single block reads) but that is as I have not re-gathered stats. I left the system to do this automatically that night.

I’ll just mention one more thing. Shrinking a table is a bit like the old trick of rebuilding indexes to compact them. People misunderstood what rebuilding indexes was achieving and why the indexes were so much larger than they “needed” to be and would spend CPU and IO rebuilding indexes overnight – only for them to quickly explode back to the original size (creating a lot of overhead as they did). Shrinking oversized tables can have a great benefit but keep in mind that if some process pushes the HWM right up every day by shoving lots of data into the table and then clearing it down later, shrinking the table is just a waste of time. It gives benefit when the table really has shrunk down in size for a while. Or it only occasionally explodes to a massive size. I have in the past written “regular shrink” scripts for tables I know explode once in a while (eg once a week) but are usually small, but you need to know what the pattern is. Don’t just blindly shrink tables.

For me in this situation, that code scanning that table dropped out of my “top 20 most demanding SQL statements” and has stayed out of it for weeks. The code now completed in considerably under 0.1 seconds as opposed to a whole second – but no one has noticed. I’d removed about 5% of the workload of the system – but no one noticed. But it took me 1/2 an hour to investigate and fix and I’d freed up a non-trivial chunk of processing resource. In my Performance Tuning heart I know I did a good job on that day.

Webinar Recording

So to wrap this up, the webinar recording (this time hosted on YouTube, the recording available from the webinar archive is hosted somewhere else) along with the webinar material for download is now officially available on AllThingsOracle.com.

I suggest that if you still have any questions regarding this webinar then you can comment on the post over there and I'll try to address them.

If you interested in more stuff like that, then stay tuned as there are more of them planned covering many of the aspects that I deliberately left out or mentioned only briefly in this basic introduction.

Extents of an Oracle Database DBA’s Knowledge Base

April 18, 2012 I saw an interesting thread on the OTN forums this morning that forced me to stop and think about several items.  The thread contains a question posed during a recent job interview – one of the best questions that I have seen (dare I say, better than one I might have crafted myself).  [...]

Wrath of the Titans…

Wrath of the Titans has one clear advantage over the previous film. It’s not a remake of a film I have seen masses of times. That alone makes it a little more interesting for me.

It looks great, but it does seem to drag on at times. I came out thinking they should have cut about 30 minutes to make it flow better. Imagine my surprise when I found out the running time was only 99 minutes. :)

IMHO this is a “wait for DVD” film. Even then I would hold on until you can get it in a sale. It’s not bad, but it won’t rock your world.

Cheers

Tim…




Battleship…

Where to begin with Battleship?

The good:

The bad:

  • If you hate disaster porn you’ll hate this film.
  • The aliens are an impressive mix of intelligent and retarded because they have some amazing technology, but they don’t bother to use it for quite obvious purposes, that would make them pretty much invincible. Lucky the human race is only ever invaded by amazingly advanced idiots.
  • The link into the Hasbro game Battleships is really ham-fisted. It’s not even similar to the game, which makes it even more embarrassing.
  • The film really needs an edit that can be shown outside the USA. There is about 10 minutes of footage that makes me want to vomit and quite frankly ruins the whole film. It makes Independence Day* feel like a multi-national experience in comparison. The fact that this is mandatory in Hollywood films kinda suggests it works will with the US audiences, so it probably pays to include it. I guess if the UK had any kind of film industry we would probably be just as bad, but since we can only make period dramas and east-end gangster films, we don’t get the opportunity to be quite so sickening. If this crap had been edited out I would have enjoyed the film sooooo much more.
  • It is so full of gaping holes you would think it was a Michael Bay film.

If you follow the blog you know that I go to some pretty awful films and enjoy quite a few of them. A little bit of editing and I would have come away saying very positive things. As it is, I’m so mentally scarred by the “Team USA” message, my overall impression was that it sucked.

Cheers

Tim…

* OMG, what a terrible film!




Critical Analysis of “Critical Analysis Meets Exadata”

Kevin Closson put out a post yesterday called Critical Analysis Meets Exadata, linking to two awesome videos. It’s well worth spending the time to watch these, even if (like me) you never get so much as a sniff of Exadata. :)

I was lucky enough to be one of several people asked to review these videos before they were released. I’m sure some of the performance gurus on the Oak Table had a lot to say, but of the several comments I fed back to Kevin, I would just like to post a couple here:

  • As a Joe Schmo dba, I almost never get to see what is happening internally in the storage layer (SAN, NAS etc). For the most part the storage is a black box that presents a few LUNs to me. If the storage subsystem and connecting network are capable of pushing enough data to and from my servers, to the point where my RAC node CPUs are the bottleneck, that is awesome. So if I think of the storage grid part of the Exadata configuration like I would judge any other SAN/NAS, then it gets a big gold star because it is good enough to keep my RAC node CPUs/cores, that are ridiculously expensive to license, working at full tilt most of the time.
  • I believe the storage cell licensing is sold on a per disk basis, not per CPU core, so the storage grid being full of idle cores does not mean I’m paying for software licensing on idle cores. If Oracle reduced the total number of CPUs/cores, the licensing costs would be unaffected. If on the other hand, the storage cells could perform a lot more of the CPU intensive load and free up the RAC nodes, then I guess the licensing situation would change, because Oracle wouldn’t want to lose those high-cost licenses from the RAC nodes.

Now Kevin is an architecture guy and I can see how from his perspective this setup sucks, because it does. He’s clearly displayed that. Then again, from a DBA perspective, do I really give a damn about some idle CPUs in the storage layer? For all I know, every other storage system out there could be just as stupid, especially now it’s impossible to by chips with small numbers of cores. :)

Like I said, you should watch the videos because they are great, but don’t be afraid to have a different opinion because you may be judging things by different standards. :)

Cheers

Tim…




Critical Analysis Meets Exadata

If you are trying to find out more in-depth information on how Exadata architecture really works, I’m offering this video presentation to offer some critical thinking on the matter. It is broken into two segments. I do recommend watching both.

Critical Analysis Meets Exadata – Part I

Critical Analysis Meets Exadata – Part II

Filed under: Exadata, Exadata Bottlenecks, oracle

Statistics poll

Thanks to everyone for spreading the link and participating in the poll on statistics. As it was pointed out in the comments, the poll isn’t very specific, I know. My point was to get an idea how many people use the defaults (I considered Oracle version 11g) for stats collection, how many do a very simple and reasonable tweak to get more stable execution plans, and how many people prefer custom statistics management. Well, I think the results will not change much and here are as following (326 votes at the time of writing):

  • 42% use the defaults (if you truncate the value. I can’t put something other than 42 here, sorry guys :-))
  • 16% don’t like unexpected histograms :-D and prefer to take situation under control
  • OMG another 42% (since it’s not Russian elections it has to be 100% in total, you know) use custom statistics management

Nice! Now we know that the defaults are suited to less than a half of Oracle installations. And my opinion is there would be more than a half using the defaults, should Oracle decided to switch the default METHOD_OPT value to ‘FOR ALL COLUMNS SIZE REPEAT’. Consider Tim Hall’s reply “On low priority systems the defaults are fine in many cases”, and think about why defaults are fine for such systems? My humble opinion is: nobody cares about low priority systems. So it doesn’t really matter if it’s reliable or not, whether there are histograms present or not, whether a system is stable or not – it’s low priority anyway. It will survive somehow. For me it means it’s not too much of a problem to go back to the future and modify default METHOD_OPT so that no histograms are gathered initially by default – as all should do manually now /me thinks.
PS. I wouldn’t mind switching GATHER_TEMP back to the FALSE and returning it to the documentation too, as it was in 9i.

Filed under: Oracle Tagged: statistics