Search

Top 60 Oracle Blogs

Recent comments

Technical

Listener Error from addNode.sh with Second Network

Recently I ran into an problem with 11.2.0.3 RAC. I observed this on a system patched to PSU6 and it looks like a bug to me. But the interesting part isn’t the problem – it’s an impressive and creative workaround that my colleague found over the weekend. I should add that this teammate doesn’t have much background with Oracle RAC though he does have lots of experience with other technologies. His email this weekend surprised me and also gave me a good laugh – hope you find it equally useful and enjoyable!

The problem originated with a requirement I was given when designing this particular cluster system: I was asked to run Data Guard traffic over the backup network instead of the public network. This sounds simple enough if you haven’t worked with RAC. But if you’ve worked with Oracle clusters you realize that nothing is simple anymore. (A big reason I often encourage people to wait on moving to RAC, especially if the main driver is high availability…)

OSP: Overview

This is the second of twelve articles in a series called Operationally Scalable Practices.  You can read the introduction in the first article. In short, this series offers helpful suggestions for younger organizations and newer DBAs to best position them for very large-scale growth.

Before getting into specifics, we will lay out a general overview of the content. I expect this overview to be revised the most as the series is refined over time – so check  periodically to see if there have been updates!

OSP: Overview

This is the second of twelve articles in a series called Operationally Scalable Practices.  You can read the introduction in the first article. In short, this series offers helpful suggestions for younger organizations and newer DBAs to best position them for very large-scale growth.

Before getting into specifics, we will lay out a general overview of the content. I expect this overview to be revised the most as the series is refined over time – so check  periodically to see if there have been updates!

Operationally Scalable Practices

I really enjoy being a technical guy. So far in my career I’ve made development choices favoring a technical path over other options. It’s been a great ride – I’ve worked in small teams and large teams; consulting roles and in-house roles; architecture/engineering roles and operations roles; big databases and little databases; environments with a few databases and environments with thousands of databases.

I’ve never been anywhere which had everything right. Also, my own ideas about what’s “right” are still evolving today. I have a habit of trying to be around people who are smarter than me… over the course of my career, my own knowledge and experience with Oracle have grown exponentially and yet I’ve never had trouble continuing to feel like a junior DBA. (In particular, my invitation to the Oak Table made this very easy!)

Operationally Scalable Practices

I really enjoy being a technical guy. So far in my career I’ve made development choices favoring a technical path over other options. It’s been a great ride – I’ve worked in small teams and large teams; consulting roles and in-house roles; architecture/engineering roles and operations roles; big databases and little databases; environments with a few databases and environments with thousands of databases.

I’ve never been anywhere which had everything right. Also, my own ideas about what’s “right” are still evolving today. I have a habit of trying to be around people who are smarter than me… over the course of my career, my own knowledge and experience with Oracle have grown exponentially and yet I’ve never had trouble continuing to feel like a junior DBA. (In particular, my invitation to the Oak Table made this very easy!)

Voting Disk Lies (CRS-4000)

Add this to the category of annoyingly unhelpful error messages.

I’m working on a mostly-automated process to create a new cluster by cloning another existing cluster. After running OUI (Oracle Universal Installer – called by config.sh to just run config assistants) there is a single ASM diskgroup which contains both the OCR and Voting Disk; however I wanted to switch the voting disks over to some different physical devices.

Upon which I received this errors:


(root)# /oracle/11203/grid/bin/crsctl replace votedisk +CLST3_VOTING
Failed to create voting files on disk group CLST3_VOTING.
Change to configuration failed, but was successfully rolled back.
CRS-4000: Command Replace failed, or completed with errors.

Lovely… so very informative. And just to be clear, it didn’t complete with errors, it completely failed. Thanks Oracle.

Voting Disk Lies (CRS-4000)

Add this to the category of annoyingly unhelpful error messages.

I’m working on a mostly-automated process to create a new cluster by cloning another existing cluster. After running OUI (Oracle Universal Installer – called by config.sh to just run config assistants) there is a single ASM diskgroup which contains both the OCR and Voting Disk; however I wanted to switch the voting disks over to some different physical devices.

Upon which I received this errors:


(root)# /oracle/11203/grid/bin/crsctl replace votedisk +CLST3_VOTING
Failed to create voting files on disk group CLST3_VOTING.
Change to configuration failed, but was successfully rolled back.
CRS-4000: Command Replace failed, or completed with errors.

Lovely… so very informative. And just to be clear, it didn’t complete with errors, it completely failed. Thanks Oracle.

Resource Manager Limits (ORA-29376)

So this is what happens when you hit that publish button too quickly – thanks Sayed for the feedback! A few corrections follow. :)


Just a brief post to mention a limit I’ve recently bumped up against. I couldn’t find any mention of this limit in the oracle docs or even on oracle’s support knowledge base. The only mention is in the text of a particular error message – I couldn’t find any further explanation or documentation.

Goal

Create a resource manager plan to guarantee fair scheduling between a large number of applications on a heavily consolidated 11.2.0.3 database. Create a consumer group for each application, put them all into a ratio-based plan so that each application gets an equal share of CPU under load.

Error

This is the error I received:

Resource Manager Limits (ORA-29376)

So this is what happens when you hit that publish button too quickly – thanks Sayed for the feedback! A few corrections follow. :)


Just a brief post to mention a limit I’ve recently bumped up against. I couldn’t find any mention of this limit in the oracle docs or even on oracle’s support knowledge base. The only mention is in the text of a particular error message – I couldn’t find any further explanation or documentation.

Goal

Create a resource manager plan to guarantee fair scheduling between a large number of applications on a heavily consolidated 11.2.0.3 database. Create a consumer group for each application, put them all into a ratio-based plan so that each application gets an equal share of CPU under load.

Error

This is the error I received:

IOUG Collaborate 2013 Wrap-Up

By now the Denver Convention Center is probably cleaned up from IOUG Collaborate. The signs directing thousands of attendees to top-notch technical presentations have been removed, the twenty rental laptops which composed the classroom for Pythian’s RAC Attack class have been returned and the vendor exhibition floor has been completely cleared out. Flight delays notwithstanding (thanks to some midwest weather), attendees are generally home by now – even those coming from places as far away as Germany and Australia.

Now that the buzz is dying down, I’ve finally found a few minutes to post my personal highlights.

Friends Old and New

First off, my favorite part of Collaborate is the opportunity to meet so many old friends and make new acquaintances who are all using Oracle technology. It’s both fun and informative to hear about the ways others are using Oracle software.