Removing objects from a read only global catalog server

[GM]

The story:

Topdomain : windows 2003
Subdomains : windows 2000 (14), windows 2003(1)
1 win2000 domain with only 1 domaincontroller crashed, active directory
restore was too old, but restored anyway (>60 days)

Problem : lingering objects only exists on 24 (read-only) global catalog
servers.
There is no writable server where these objects anymore.

On win2003 I can remove the objects temporarely with repadmin
/removelingeringobjects (until replication occurs, then they are back)

I created a script as described in : http://support.microsoft.com/kb/314282
Script gets error -2147016672 on all servers (because they are all
read-only)

I tried deleting the object using vbscript (but this does not work)

dim ouser
set ouser = getobject("GC://hermes:3268/CN=name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a,OU=some
OU,DC=subdomain,DC=domain,DC=net")
ouser.DeleteObject(0)

I tried creating a new user on the subdomain and then renaming it to "name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a" using vbscript, but this
does not work (can't use the special character \0A)

I upgraded the original crashed domain to win2003 too see if it would make
any difference, but problem remains.

Maybe there is a way to rebuild all GC's from scratch all at once?

Repadmin /removelingeringobjects can delete objects on read-only global
catalogs (but not on win2000)
If I know which api it uses and if this api exists on win2000, then I can
try to create a script to delete them all at once on all GC's.

Ideas are welcome.

[Austin Osuide]

Hi GM,
A few issues here I think:
1. A GC may have read only partitions and 1 RW partition for its domain.
2. You cannot remove a lingering object from a read-only partition on a GC
with repadmin. It only has a read copy of the partition replicated from a DC
authoritative for that partition and hence the error in your script.
I think you may need to concentrate on the DC that has the lingering object
and delete it frm there and let replication do its thing.
Read http://support.microsoft.com/kb/314282 again and follow steps.
Also, if you still have issues, perhaps posting the Event log entries
indicating ligering objects?

Regards,

Austin

Post by GM
Topdomain : windows 2003
Subdomains : windows 2000 (14), windows 2003(1)
1 win2000 domain with only 1 domaincontroller crashed, active directory
restore was too old, but restored anyway (>60 days)
Problem : lingering objects only exists on 24 (read-only) global catalog
servers.
There is no writable server where these objects anymore.
On win2003 I can remove the objects temporarely with repadmin
/removelingeringobjects (until replication occurs, then they are back)
I created a script as described in : http://support.microsoft.com/kb/314282
Script gets error -2147016672 on all servers (because they are all
read-only)
I tried deleting the object using vbscript (but this does not work)
dim ouser
set ouser = getobject("GC://hermes:3268/CN=name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a,OU=some
OU,DC=subdomain,DC=domain,DC=net")
ouser.DeleteObject(0)
I tried creating a new user on the subdomain and then renaming it to "name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a" using vbscript, but this
does not work (can't use the special character \0A)
I upgraded the original crashed domain to win2003 too see if it would make
any difference, but problem remains.
Maybe there is a way to rebuild all GC's from scratch all at once?
Repadmin /removelingeringobjects can delete objects on read-only global
catalogs (but not on win2000)
If I know which api it uses and if this api exists on win2000, then I can
try to create a script to delete them all at once on all GC's.
Ideas are welcome.

[GM]

The problem is that there are ONLY R partitions for this object, no RW, this
is because the restored domain controller older than 60 days was the only
domain controller in this (sub)domain.

The result = the lingering object doesn't exist in the domain (the RW
partition), only on all the read partitions in all GC's in the other
domains.

So as I see it, I need a way to remove from the Read partition (because the
object doens't exist in the Write partition)

Post by Austin Osuide
Hi GM,
1. A GC may have read only partitions and 1 RW partition for its domain.
2. You cannot remove a lingering object from a read-only partition on a GC
with repadmin. It only has a read copy of the partition replicated from a DC
authoritative for that partition and hence the error in your script.
I think you may need to concentrate on the DC that has the lingering object
and delete it frm there and let replication do its thing.
Read http://support.microsoft.com/kb/314282 again and follow steps.
Also, if you still have issues, perhaps posting the Event log entries
indicating ligering objects?
Regards,
Austin

Post by GM
Topdomain : windows 2003
Subdomains : windows 2000 (14), windows 2003(1)
1 win2000 domain with only 1 domaincontroller crashed, active directory
restore was too old, but restored anyway (>60 days)
Problem : lingering objects only exists on 24 (read-only) global catalog
servers.
There is no writable server where these objects anymore.
On win2003 I can remove the objects temporarely with repadmin
/removelingeringobjects (until replication occurs, then they are back)
http://support.microsoft.com/kb/314282
Script gets error -2147016672 on all servers (because they are all
read-only)
I tried deleting the object using vbscript (but this does not work)
dim ouser
set ouser = getobject("GC://hermes:3268/CN=name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a,OU=some
OU,DC=subdomain,DC=domain,DC=net")
ouser.DeleteObject(0)
I tried creating a new user on the subdomain and then renaming it to "name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a" using vbscript, but this
does not work (can't use the special character \0A)
I upgraded the original crashed domain to win2003 too see if it would make
any difference, but problem remains.
Maybe there is a way to rebuild all GC's from scratch all at once?
Repadmin /removelingeringobjects can delete objects on read-only global
catalogs (but not on win2000)
If I know which api it uses and if this api exists on win2000, then I can
try to create a script to delete them all at once on all GC's.
Ideas are welcome.

[Tim Chin]

I had this happen once before when replication wasn't working for over the
tombstone period. Since Windows 2000 was involved, I couldn't run repadmin
/removelingeringobjects and was forced to turn off all Global Catalogs in
the entire forest. Here's what I did:

Initiate all GC removes at once
Reboot every server in staggered intervals
Initiate and wait for a forest level GC build to complete
Initiate all remaining GC builds

All removes must be complete before building. Otherwise, you'll end up with
lingering objects again. This was a very scary process at the time, but as
long as you know the builtin\administrator password the forest domain and
have replication configured properly, there's not much to worry about.

--
Tim

Post by GM
The problem is that there are ONLY R partitions for this object, no RW,
this is because the restored domain controller older than 60 days was the
only domain controller in this (sub)domain.
The result = the lingering object doesn't exist in the domain (the RW
partition), only on all the read partitions in all GC's in the other
domains.
So as I see it, I need a way to remove from the Read partition (because
the object doens't exist in the Write partition)

[Austin Osuide]

Hi GM,
To me, the options you have are the following:

1. Build a new forest and migrate (Don't hit me, I'm serious!)
2. UnGC and GC all the global catalogs one at a time,
or
3. Rehost all R partitions on all GCs using repadmin /rehost

I'd try to UnGC the DCs first and try and rebuild the read only partitions,
the the rehosting of all RO partitions before embarking on a forest rebuild

Regards,

Austin

Post by GM
The problem is that there are ONLY R partitions for this object, no RW,
this is because the restored domain controller older than 60 days was the
only domain controller in this (sub)domain.
The result = the lingering object doesn't exist in the domain (the RW
partition), only on all the read partitions in all GC's in the other
domains.
So as I see it, I need a way to remove from the Read partition (because
the object doens't exist in the Write partition)

Post by Austin Osuide
Hi GM,
1. A GC may have read only partitions and 1 RW partition for its domain.
2. You cannot remove a lingering object from a read-only partition on a GC
with repadmin. It only has a read copy of the partition replicated from a DC
authoritative for that partition and hence the error in your script.
I think you may need to concentrate on the DC that has the lingering object
and delete it frm there and let replication do its thing.
Read http://support.microsoft.com/kb/314282 again and follow steps.
Also, if you still have issues, perhaps posting the Event log entries
indicating ligering objects?
Regards,
Austin

Post by GM
Topdomain : windows 2003
Subdomains : windows 2000 (14), windows 2003(1)
1 win2000 domain with only 1 domaincontroller crashed, active directory
restore was too old, but restored anyway (>60 days)
Problem : lingering objects only exists on 24 (read-only) global catalog
servers.
There is no writable server where these objects anymore.
On win2003 I can remove the objects temporarely with repadmin
/removelingeringobjects (until replication occurs, then they are back)
http://support.microsoft.com/kb/314282
Script gets error -2147016672 on all servers (because they are all
read-only)
I tried deleting the object using vbscript (but this does not work)
dim ouser
set ouser = getobject("GC://hermes:3268/CN=name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a,OU=some
OU,DC=subdomain,DC=domain,DC=net")
ouser.DeleteObject(0)
I tried creating a new user on the subdomain and then renaming it to "name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a" using vbscript, but this
does not work (can't use the special character \0A)
I upgraded the original crashed domain to win2003 too see if it would make
any difference, but problem remains.
Maybe there is a way to rebuild all GC's from scratch all at once?
Repadmin /removelingeringobjects can delete objects on read-only global
catalogs (but not on win2000)
If I know which api it uses and if this api exists on win2000, then I can
try to create a script to delete them all at once on all GC's.
Ideas are welcome.

[GM]

These are indeed the options I already found :(
2) is tricky because of the replication, wouldn't it be better to ungc all
gc's at once and then GC them one at a time?
3) Rehosting : I read this somewhere before, but I don't understand how this
can solve the problem.

I think 2 is maybe the easiest way to go?, can be done in less than 24 hours
probably.

The way I see it : Active directory is just a database, you can
delete/add/change records on it (but in this case the windows API used is
denying it)
I know that ntdsutil can delete servers from a read-only partition, so there
must be a way (secret api/offline editing/dll injection, ...) to do it

Post by Austin Osuide
Hi GM,
1. Build a new forest and migrate (Don't hit me, I'm serious!)
2. UnGC and GC all the global catalogs one at a time,
or
3. Rehost all R partitions on all GCs using repadmin /rehost
I'd try to UnGC the DCs first and try and rebuild the read only partitions,
the the rehosting of all RO partitions before embarking on a forest rebuild
Regards,
Austin

Post by GM
The problem is that there are ONLY R partitions for this object, no RW,
this is because the restored domain controller older than 60 days was the
only domain controller in this (sub)domain.
The result = the lingering object doesn't exist in the domain (the RW
partition), only on all the read partitions in all GC's in the other
domains.
So as I see it, I need a way to remove from the Read partition (because
the object doens't exist in the Write partition)

Post by Austin Osuide
Hi GM,
1. A GC may have read only partitions and 1 RW partition for its domain.
2. You cannot remove a lingering object from a read-only partition on a GC
with repadmin. It only has a read copy of the partition replicated from a DC
authoritative for that partition and hence the error in your script.
I think you may need to concentrate on the DC that has the lingering object
and delete it frm there and let replication do its thing.
Read http://support.microsoft.com/kb/314282 again and follow steps.
Also, if you still have issues, perhaps posting the Event log entries
indicating ligering objects?
Regards,
Austin

Post by GM
Topdomain : windows 2003
Subdomains : windows 2000 (14), windows 2003(1)
1 win2000 domain with only 1 domaincontroller crashed, active directory
restore was too old, but restored anyway (>60 days)
Problem : lingering objects only exists on 24 (read-only) global catalog
servers.
There is no writable server where these objects anymore.
On win2003 I can remove the objects temporarely with repadmin
/removelingeringobjects (until replication occurs, then they are back)
http://support.microsoft.com/kb/314282
Script gets error -2147016672 on all servers (because they are all
read-only)
I tried deleting the object using vbscript (but this does not work)
dim ouser
set ouser = getobject("GC://hermes:3268/CN=name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a,OU=some
OU,DC=subdomain,DC=domain,DC=net")
ouser.DeleteObject(0)
I tried creating a new user on the subdomain and then renaming it to "name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a" using vbscript, but this
does not work (can't use the special character \0A)
I upgraded the original crashed domain to win2003 too see if it would make
any difference, but problem remains.
Maybe there is a way to rebuild all GC's from scratch all at once?
Repadmin /removelingeringobjects can delete objects on read-only global
catalogs (but not on win2000)
If I know which api it uses and if this api exists on win2000, then I can
try to create a script to delete them all at once on all GC's.
Ideas are welcome.

[Austin Osuide]

Hi GM,

If what you say is correct, the RW replica set for the domain NC is
consistent. It is the GCs, which have RO replicas for that Domain that are
"in a bad way".
So, what you want to do is to re-source that partition from a writable.
Since "all RW are consistent" we look good to go:

1. unGC the DC. It gracefully looses the RO partition.
2. Re-GC the DC (again, because I believe you here when you say the writable
are clean, we do not run the risk of resourcing the lingering objects and we
can unGC one DC at a time.)

WRT the Repadmain /rehost,
You can also do this as you seem to be v.sure the RW is clean.
the syntax for /rehost is:
repadmin /rehost DC <Naming Context> <Good Source DC Address>
[/application]
If a source DC Address is not specified, a writeable replica will be
selected using the DC Locator

In this case though, I would try to isolate the GC while this was being done
so it doesn't continue to service clients during the process.

Regards,

Austin

Post by GM
These are indeed the options I already found :(
2) is tricky because of the replication, wouldn't it be better to ungc all
gc's at once and then GC them one at a time?
3) Rehosting : I read this somewhere before, but I don't understand how
this can solve the problem.
I think 2 is maybe the easiest way to go?, can be done in less than 24
hours probably.
The way I see it : Active directory is just a database, you can
delete/add/change records on it (but in this case the windows API used is
denying it)
I know that ntdsutil can delete servers from a read-only partition, so
there must be a way (secret api/offline editing/dll injection, ...) to do
it

Post by Austin Osuide
Hi GM,
1. Build a new forest and migrate (Don't hit me, I'm serious!)
2. UnGC and GC all the global catalogs one at a time,
or
3. Rehost all R partitions on all GCs using repadmin /rehost
I'd try to UnGC the DCs first and try and rebuild the read only partitions,
the the rehosting of all RO partitions before embarking on a forest rebuild
Regards,
Austin

Post by GM
The problem is that there are ONLY R partitions for this object, no RW,
this is because the restored domain controller older than 60 days was
the only domain controller in this (sub)domain.
The result = the lingering object doesn't exist in the domain (the RW
partition), only on all the read partitions in all GC's in the other
domains.
So as I see it, I need a way to remove from the Read partition (because
the object doens't exist in the Write partition)

Post by Austin Osuide
Hi GM,
1. A GC may have read only partitions and 1 RW partition for its domain.
2. You cannot remove a lingering object from a read-only partition on a GC
with repadmin. It only has a read copy of the partition replicated from a DC
authoritative for that partition and hence the error in your script.
I think you may need to concentrate on the DC that has the lingering object
and delete it frm there and let replication do its thing.
Read http://support.microsoft.com/kb/314282 again and follow steps.
Also, if you still have issues, perhaps posting the Event log entries
indicating ligering objects?
Regards,
Austin

Post by GM
Topdomain : windows 2003
Subdomains : windows 2000 (14), windows 2003(1)
1 win2000 domain with only 1 domaincontroller crashed, active directory
restore was too old, but restored anyway (>60 days)
Problem : lingering objects only exists on 24 (read-only) global catalog
servers.
There is no writable server where these objects anymore.
On win2003 I can remove the objects temporarely with repadmin
/removelingeringobjects (until replication occurs, then they are back)
http://support.microsoft.com/kb/314282
Script gets error -2147016672 on all servers (because they are all
read-only)
I tried deleting the object using vbscript (but this does not work)
dim ouser
set ouser = getobject("GC://hermes:3268/CN=name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a,OU=some
OU,DC=subdomain,DC=domain,DC=net")
ouser.DeleteObject(0)
I tried creating a new user on the subdomain and then renaming it to "name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a" using vbscript, but this
does not work (can't use the special character \0A)
I upgraded the original crashed domain to win2003 too see if it would make
any difference, but problem remains.
Maybe there is a way to rebuild all GC's from scratch all at once?
Repadmin /removelingeringobjects can delete objects on read-only global
catalogs (but not on win2000)
If I know which api it uses and if this api exists on win2000, then I can
try to create a script to delete them all at once on all GC's.
Ideas are welcome.

[nicolas grillon]

Hi,

Have you tried this command :

REPADMIN /REBUILDGC <CONTROLEUR_NAME>

This command rebuild the GC from the NC of the childs or from the local
NC.
I use this for delete the lingering objects of my Forest GC.

nicolas grillon

[Jorge de Almeida Pinto [MVP - DS]]

http://blogs.dirteam.com/blogs/jorge/archive/2006/05/08/Lingering-objects.aspx
http://blogs.dirteam.com/blogs/jorge/archive/2005/11/24/153.aspx
--
Cheers,
(HOPEFULLY THIS INFORMATION HELPS YOU!)

# Jorge de Almeida Pinto # MVP Windows Server - Directory Services

BLOG (WEB-BASED)--> http://blogs.dirteam.com/blogs/jorge/default.aspx
BLOG (RSS-FEEDS)--> http://blogs.dirteam.com/blogs/jorge/rss.aspx
------------------------------------------------------------------------------------------
* How to ask a question --> http://support.microsoft.com/?id=555375
------------------------------------------------------------------------------------------
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always test before implementing!
------------------------------------------------------------------------------------------
#################################################
#################################################
------------------------------------------------------------------------------------------

Post by nicolas grillon
Hi,
REPADMIN /REBUILDGC <CONTROLEUR_NAME>
This command rebuild the GC from the NC of the childs or from the local
NC.
I use this for delete the lingering objects of my Forest GC.
nicolas grillon
--
nicolas grillon
------------------------------------------------------------------------
http://forums.techarena.in/member.php?userid=39728
View this thread: http://forums.techarena.in/showthread.php?t=832643
http://forums.techarena.in

[Jorge de Almeida Pinto [MVP - DS]]

Post by GM

Post by GM
There is no writable server where these objects anymore.

what do you mean with this?

Post by GM
On win2003 I can remove the objects temporarely with repadmin
/removelingeringobjects (until replication occurs, then they are back)

then you must have LOOSE replication enabled (not recommended). STRICT
replication is the way to go!

Post by GM
removing lingering objects

on w2k3 GCs you can use REPADMIN
on w2k GC you must use the VBS script as mentioned in the KB

also see:
http://blogs.dirteam.com/blogs/jorge/archive/2006/05/08/Lingering-objects.aspx

Post by GM
Topdomain : windows 2003
Subdomains : windows 2000 (14), windows 2003(1)
1 win2000 domain with only 1 domaincontroller crashed, active directory
restore was too old, but restored anyway (>60 days)
Problem : lingering objects only exists on 24 (read-only) global catalog
servers.
There is no writable server where these objects anymore.
On win2003 I can remove the objects temporarely with repadmin
/removelingeringobjects (until replication occurs, then they are back)
http://support.microsoft.com/kb/314282
Script gets error -2147016672 on all servers (because they are all
read-only)
I tried deleting the object using vbscript (but this does not work)
dim ouser
set ouser = getobject("GC://hermes:3268/CN=name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a,OU=some
OU,DC=subdomain,DC=domain,DC=net")
ouser.DeleteObject(0)
I tried creating a new user on the subdomain and then renaming it to "name
name\0ACNF:7899398a-5697-472f-8db8-0703926c5c3a" using vbscript, but this
does not work (can't use the special character \0A)
I upgraded the original crashed domain to win2003 too see if it would make
any difference, but problem remains.
Maybe there is a way to rebuild all GC's from scratch all at once?
Repadmin /removelingeringobjects can delete objects on read-only global
catalogs (but not on win2000)
If I know which api it uses and if this api exists on win2000, then I can
try to create a script to delete them all at once on all GC's.
Ideas are welcome.