Adding index to attribute in AD

[Clive George]

What are the potential pitfalls of indexing eg extensionAttribute3 in our AD
schema? (it's used to hold employee number, and I'd like to query by
employee number, so indexing this would seem to be sensible).

Our AD administrator is not wanting to do such a thing, so I thought I'd ask
the general world whether this is normal or not.

(I'm assuming indexing is setting searchFlags bit 0 to 1 in the schema.
Also, this is straight AD, not ADAM.)

cheers,
clive

[Al Mulnick]

Are you planning to programmatically make this change? Why not via the
schema admin?

http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/ServerHelp/44cb16a2-7969-4e59-8c9b-5962604ff1fb.mspx

Setting it to 1 from it's current may or may not be the right answer.
That's a field that can have the values in the following table:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/adschema/adschema/a_searchflags.asp

To index, it must be at least 1 but could be a combination.

Not sure what exactly to tell you about this. If the internal AD admin
doesn't want to do it, I have to ask why before I can make a suggestion.
That person may have a very good reason.




Al

Post by Clive George
What are the potential pitfalls of indexing eg extensionAttribute3 in our
AD schema? (it's used to hold employee number, and I'd like to query by
employee number, so indexing this would seem to be sensible).
Our AD administrator is not wanting to do such a thing, so I thought I'd
ask the general world whether this is normal or not.
(I'm assuming indexing is setting searchFlags bit 0 to 1 in the schema.
Also, this is straight AD, not ADAM.)
cheers,
clive

[Clive George]

I'm not concerned about the 'how' - I wouldn't be the one making the change.
I only mentioned searchFlags bit 0 to avoid confusion (I appear to have
failed slightly :-) ).

What I would like to know is 'why not'.

Reasons could include performance, ease of future administration.

Somebody's hinted that if an attribute isn't actually unique, the index
won't get used - is this true? Seems odd to me.

cheers,
clive

Post by Al Mulnick
Are you planning to programmatically make this change? Why not via the
schema admin?
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/ServerHelp/44cb16a2-7969-4e59-8c9b-5962604ff1fb.mspx
Setting it to 1 from it's current may or may not be the right answer.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/adschema/adschema/a_searchflags.asp
To index, it must be at least 1 but could be a combination.
Not sure what exactly to tell you about this. If the internal AD admin
doesn't want to do it, I have to ask why before I can make a suggestion.
That person may have a very good reason.
Al

Post by Clive George
What are the potential pitfalls of indexing eg extensionAttribute3 in our
AD schema? (it's used to hold employee number, and I'd like to query by
employee number, so indexing this would seem to be sensible).
Our AD administrator is not wanting to do such a thing, so I thought I'd
ask the general world whether this is normal or not.
(I'm assuming indexing is setting searchFlags bit 0 to 1 in the schema.
Also, this is straight AD, not ADAM.)
cheers,
clive

[Al Mulnick]

Not sure of the situation where they mean an attribute is not unique. Maybe
the value?

Usual reasons why not:
1) increased storage space to hold the global catalog information (if it's
not already in the GC)
2) increased overhead to process the index
3) not sure why else somebody wouldn't? Personal reasons?

Future administration should be unaffected. Performance is increased for the
searches if that's what you're doing. Machines would have to work slightly
harder as the index increases but that's normal behavior so I doubt it would
be much for one attribute to be indexed.

Does that help or are you looking for something else?

Post by Clive George
I'm not concerned about the 'how' - I wouldn't be the one making the
change. I only mentioned searchFlags bit 0 to avoid confusion (I appear to
have failed slightly :-) ).
What I would like to know is 'why not'.
Reasons could include performance, ease of future administration.
Somebody's hinted that if an attribute isn't actually unique, the index
won't get used - is this true? Seems odd to me.
cheers,
clive

Post by Al Mulnick
Are you planning to programmatically make this change? Why not via the
schema admin?
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/ServerHelp/44cb16a2-7969-4e59-8c9b-5962604ff1fb.mspx
Setting it to 1 from it's current may or may not be the right answer.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/adschema/adschema/a_searchflags.asp
To index, it must be at least 1 but could be a combination.
Not sure what exactly to tell you about this. If the internal AD admin
doesn't want to do it, I have to ask why before I can make a suggestion.
That person may have a very good reason.
Al

Post by Clive George
What are the potential pitfalls of indexing eg extensionAttribute3 in
our AD schema? (it's used to hold employee number, and I'd like to query
by employee number, so indexing this would seem to be sensible).
Our AD administrator is not wanting to do such a thing, so I thought I'd
ask the general world whether this is normal or not.
(I'm assuming indexing is setting searchFlags bit 0 to 1 in the schema.
Also, this is straight AD, not ADAM.)
cheers,
clive

[Clive George]

Sorry, yes, I meant an attribute value is not unique. Eg we manage to get
two people with the same staff number - would this stop it indexing on staff
number?

How bad is 1 and 2? The data in the attribute is a 5 character string, and
there are about 5000 users. The only change would be to index this
attribute.

There are currently 22 indexed attributes for a person - how much will one
extra hit performance?

As you may have guessed, I'm looking for ways to persuade our AD admins. Or
me, if it turns out that changing schemas in this way is not 'the done
thing'.

There were also hints that MSCE training said one should avoid adding extra
indexing, but no reasons for this have been put forward - anybody know any?

cheers,
clive

Post by Al Mulnick
Not sure of the situation where they mean an attribute is not unique.
Maybe the value?
1) increased storage space to hold the global catalog information (if it's
not already in the GC)
2) increased overhead to process the index
3) not sure why else somebody wouldn't? Personal reasons?
Future administration should be unaffected. Performance is increased for
the searches if that's what you're doing. Machines would have to work
slightly harder as the index increases but that's normal behavior so I
doubt it would be much for one attribute to be indexed.
Does that help or are you looking for something else?

Post by Clive George
I'm not concerned about the 'how' - I wouldn't be the one making the
change. I only mentioned searchFlags bit 0 to avoid confusion (I appear
to have failed slightly :-) ).
What I would like to know is 'why not'.
Reasons could include performance, ease of future administration.
Somebody's hinted that if an attribute isn't actually unique, the index
won't get used - is this true? Seems odd to me.
cheers,
clive

Post by Al Mulnick
Are you planning to programmatically make this change? Why not via the
schema admin?
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/ServerHelp/44cb16a2-7969-4e59-8c9b-5962604ff1fb.mspx
Setting it to 1 from it's current may or may not be the right answer.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/adschema/adschema/a_searchflags.asp
To index, it must be at least 1 but could be a combination.
Not sure what exactly to tell you about this. If the internal AD admin
doesn't want to do it, I have to ask why before I can make a suggestion.
That person may have a very good reason.
Al

Post by Clive George
What are the potential pitfalls of indexing eg extensionAttribute3 in
our AD schema? (it's used to hold employee number, and I'd like to
query by employee number, so indexing this would seem to be sensible).
Our AD administrator is not wanting to do such a thing, so I thought
I'd ask the general world whether this is normal or not.
(I'm assuming indexing is setting searchFlags bit 0 to 1 in the schema.
Also, this is straight AD, not ADAM.)
cheers,
clive

[Al Mulnick]

No, based on the information you present, you should index the value. That's
what indexing is for and why it's configurable; to speed up searches based
on that attribute. I will say that it's possible that there's a better way
to do what you're doing, but based on the currently available information, I
can't think of anything helpful other than to index the attribute.

"Sorry, yes, I meant an attribute value is not unique. Eg we manage to get
two people with the same staff number - would this stop it indexing on staff
number?"

No, it won't stop indexing, but it will have to return more than one result
right? I've never heard of the indexing stopping, but in this case it would
be less effective because indexing should be used for unique values to
really be effective. However, that is not always the case as in some of the
other 22 fields that are indexed which can be unique or not by themselves.
When you search on an indexed field, you'd expect to get multiple hits for
this type of query.

You don't want to turn on indexing willy-nilly for all of your attributes.
There are costs associated often in DIT expansion and replication. However,
for 5000 user objects (that's what? One group worth?) the extra burden is
likely next to nothing. I'm not even convinced you'd notice a difference in
the day to day. Even if you have a lot of DC's and they're distributed,
indexing that attribute for 5000 objects isn't likely to be noticed in any
perceptible manner.

Your queries would get faster, but not likely be a tremendous amount.

Al

Post by Clive George
Sorry, yes, I meant an attribute value is not unique. Eg we manage to get
two people with the same staff number - would this stop it indexing on
staff number?
How bad is 1 and 2? The data in the attribute is a 5 character string, and
there are about 5000 users. The only change would be to index this
attribute.
There are currently 22 indexed attributes for a person - how much will one
extra hit performance?
As you may have guessed, I'm looking for ways to persuade our AD admins.
Or me, if it turns out that changing schemas in this way is not 'the done
thing'.
There were also hints that MSCE training said one should avoid adding
extra indexing, but no reasons for this have been put forward - anybody
know any?
cheers,
clive

Post by Al Mulnick
Not sure of the situation where they mean an attribute is not unique.
Maybe the value?
1) increased storage space to hold the global catalog information (if
it's not already in the GC)
2) increased overhead to process the index
3) not sure why else somebody wouldn't? Personal reasons?
Future administration should be unaffected. Performance is increased for
the searches if that's what you're doing. Machines would have to work
slightly harder as the index increases but that's normal behavior so I
doubt it would be much for one attribute to be indexed.
Does that help or are you looking for something else?

Post by Clive George
I'm not concerned about the 'how' - I wouldn't be the one making the
change. I only mentioned searchFlags bit 0 to avoid confusion (I appear
to have failed slightly :-) ).
What I would like to know is 'why not'.
Reasons could include performance, ease of future administration.
Somebody's hinted that if an attribute isn't actually unique, the index
won't get used - is this true? Seems odd to me.
cheers,
clive

Post by Al Mulnick
Are you planning to programmatically make this change? Why not via
the schema admin?
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/ServerHelp/44cb16a2-7969-4e59-8c9b-5962604ff1fb.mspx
Setting it to 1 from it's current may or may not be the right answer.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/adschema/adschema/a_searchflags.asp
To index, it must be at least 1 but could be a combination.
Not sure what exactly to tell you about this. If the internal AD admin
doesn't want to do it, I have to ask why before I can make a
suggestion. That person may have a very good reason.
Al

Post by Clive George
What are the potential pitfalls of indexing eg extensionAttribute3 in
our AD schema? (it's used to hold employee number, and I'd like to
query by employee number, so indexing this would seem to be sensible).
Our AD administrator is not wanting to do such a thing, so I thought
I'd ask the general world whether this is normal or not.
(I'm assuming indexing is setting searchFlags bit 0 to 1 in the
schema. Also, this is straight AD, not ADAM.)
cheers,
clive

[Clive George]

"Al Mulnick" <***@ncDOTrr.com> wrote in message news:***@tk2msftngp13.phx.gbl...

<helpful stuff>

Thanks for that Al. It's pretty much what I thought. Now I've got to go and
persuade people...

(and do some performance testing to see if it's necessary - I thought I saw
a difference when using ADAM though).

cheers,
clive

[Clive George]

Further to this:

Our AD administrator has mentioned that he's concerned that schema changes,
including creating indexes, can't be rolled back. Is this true for indexes?
Ie can you unset the relevant flag?

cheers,
clive

[Al Mulnick]

Schema changes can't be undone per se. Indexes can be set and unset.
That's the not the same as the schema changes being thought of here i.e. you
add a new objectclass which you can't later undo. You can however
invalidate that schema object.

The index setting is not the same thing. It's there to allow you to
increase performance for searches and should be employed in such situations
where you need to make frequent searches. It actually helps to reduce the
impact of the searches against the domain controllers and is how the system
is designed (most LDAP directories have similar feature to improve
performance).

Al

Post by Clive George
Our AD administrator has mentioned that he's concerned that schema
changes, including creating indexes, can't be rolled back. Is this true
for indexes? Ie can you unset the relevant flag?
cheers,
clive

[Joe Richards [MVP]]

As Al mentioned, the issue is in removing added attributes and objectclasses.
You can only defunct them, not completely delete them. However defuncting is
about as good as a delete so it really shouldn't matter.

Things like indexing and PAS membership, etc can easily be done and undone.

--
Joe Richards Microsoft MVP Windows Server Directory Services
www.joeware.net

Post by Clive George
Our AD administrator has mentioned that he's concerned that schema changes,
including creating indexes, can't be rolled back. Is this true for indexes?
Ie can you unset the relevant flag?
cheers,
clive

[Joe Richards [MVP]]

Indexing is normally quite safe, you just want to be careful with how many
attributes you index because you can start to grow the DIT considerably. There
will be a little bit of a perf hit for object insertions, this will get you on
new object adds and anytime the attribute in question is modified but it usually
isn't even noticable.

You can not effectively index linked attributes (such as member/memberof) but
that is ok because they are implicitely indexed, however you don't actually get
to take advantage of that until K3 AD. 2K isn't smart enough to use the index.

Way back in the day there was concern over indexing non-unique or multivalue
attributes. This is NOT a problem. I often recommend companies to index
objectclass to speed up poorly written AD applications such as Exchange and
other apps and objectclass is very non-unique and quite multi-valued and you get
considerable benefit for usually less than a 10% DIT expansion for doing it. I
have seen emails that hint that some knowledgable people inside of MS think that
not indexing objectclass was a mistake.

Anyway, don't index something you search on once in a blue moon, if it is
something you are searching on quite a bit every day, look at indexing.

joe





--
Joe Richards Microsoft MVP Windows Server Directory Services
www.joeware.net

Post by Clive George
What are the potential pitfalls of indexing eg extensionAttribute3 in our AD
schema? (it's used to hold employee number, and I'd like to query by
employee number, so indexing this would seem to be sensible).
Our AD administrator is not wanting to do such a thing, so I thought I'd ask
the general world whether this is normal or not.
(I'm assuming indexing is setting searchFlags bit 0 to 1 in the schema.
Also, this is straight AD, not ADAM.)
cheers,
clive

[Clive George]

"Joe Richards [MVP]" <***@hotmail.com> wrote in message news:Om6$***@TK2MSFTNGP09.phx.gbl...

<useful stuff>

Thanks for the reply. Seems to be pretty much the same as Al wrote, so it's
good to know there's more than one person with this opinion.

cheers,
clive

[Jay Armstrong]

Clive,

Sorry to be so late inputing, I read this last week and was called to a
meeting...

In our enterprise we use the employeeID attribute to store data on our
users. We currently have >200,000 student accounts with unique IDs and >500
accounts that fall into categories (i.e. Instructor, Service, ITStaff, etc)
that are not unique.

I use this field for lookups on the students (the user name may change, but
the unique ID follows them forever) and to filter logins to a web portal (the
student id is always numeric, Instructors get one site, ITStaff another,
etc). Before indexing the employeeID attribute the LDAP searches took approx.
20-25 seconds on average and often timed out. After indexing the searches
complete in less than a second.

We have seen no appreciable increase in the ntds.dit file from the schema
mod and since the field is static after creation replication has not been
affected.

So, to echo the others with some real world examples, there should be no
problem with indexing the field in my experiance.

Jay

Post by Clive George
What are the potential pitfalls of indexing eg extensionAttribute3 in our AD
schema? (it's used to hold employee number, and I'd like to query by
employee number, so indexing this would seem to be sensible).
Our AD administrator is not wanting to do such a thing, so I thought I'd ask
the general world whether this is normal or not.
(I'm assuming indexing is setting searchFlags bit 0 to 1 in the schema.
Also, this is straight AD, not ADAM.)
cheers,
clive