Hey thanks JC (and others)....

I'll have to dust off my C++ skills (and compiler) and try these things...

I'll report back to the group on any of my findings, and any other suggestions are more than welcome.

Cheers,

Keith


> JC Ferguson <jc@F5.com> wrote:
> 
> 
> There is one poor design choice made, the way I see it.  If you look at 
> lsaext.c, it appears to do this, for EACH user:
> 
> - SendUserData
> - Encrypt
> - WriteData to the pipe
> - Flush the pipe
> - then reset it, i.e., this code:
> 
> 	if (hPipe != NULL && hPipe != INVALID_HANDLE_VALUE)
> 	{
> 		WriteFile(hPipe, szData, sizeof(USERINFO) + 3, &cbTotalBytes, NULL);
> 		FlushFileBuffers(hPipe); 
> 		ResetPipe();
> 	}
> 
> ResetPipe() is this code:
> 
> BOOL ResetPipe()
> {
> 	if (hPipe != NULL && hPipe != INVALID_HANDLE_VALUE)
> 	{
> 		DisconnectNamedPipe(hPipe);
> 		return ConnectNamedPipe(hPipe, NULL) ? TRUE : (GetLastError() == 
> ERROR_PIPE_CONNECTED || GetLastError() == ERROR_PIPE_LISTENING); 
> 	}
> 	else
> 		return FALSE;
> }
> 
> 
> So, for EACH user, the pipe is disconnected, then reconnected.  That is 
> an incredibly expensive operation in the Windows kernel, especially for 
> large AD environments, for the purpose of sending a tiny amount of data, 
> i.e., less than 512 bytes.  You have 3 million users in your AD, you are 
> doing that operation 3 million times.  Unless I am missing something in 
> reading the code...
> 
> And, I don't quite get why that design choice was made - named pipes , 
> on a local machine especially, are pretty reliable.
> 
> I betcha, if you remove the ResetPipe() stuff, you will see dramatic 
> improvements.
> 
> 
> -jc
> 
> 
> > -----Original Message-----
> > From: foofus-tools-bounces@lists.foofus.net [mailto:foofus-tools-
> > bounces@lists.foofus.net] On Behalf Of Keith Morrell
> > Sent: Wednesday, May 19, 2010 17:14
> > To: fizzgig@foofus.net; foofus-tools@lists.foofus.net
> > Subject: Re: [foofus-tools] pwdump6
> > 
> > Hey, thanks! Sounds great...I'd be keen to get a beta copy and give it 
> a try
> > for you.
> > 
> > But I was wondering (not that I know much about AD internals) how you 
> would
> > pull in the accounts from the registry? I thought all the info would 
> be in the
> > NTDS\dit file?
> > 
> > Also, I'm not worried if it takes a few hours to run, just that it 
> completes
> > AND produces some output...and I was wondering if there is any size 
> limit
> > "built-in" that would prevent it working (some array limit or some 
> such
> > "programmer" type limit?)
> > 
> > As it stands now, it takes maybe 3-4 hours and looks to have 
> completed...but
> > no output.
> > 
> > As Thomas mentioned, a setting could be made 
> ""pSamrEnumerateUsersInDomain"
> > function inside the "LsaExt.c" from 0 to 0x10" to exclude 
> system/computer
> > accounts, which is fine by me as I only need to export user account 
> hashes.
> > 
> > Thanks for your help!
> > 
> > Cheers,
> > 
> > Keith
> > 
> > -----Original Message-----
> > From: Teh Fizzgig [mailto:fizzgig@foofus.net]
> > Sent: Wednesday, 19 May 2010 7:55 AM
> > To: foofus-tools@lists.foofus.net
> > Subject: Re: [foofus-tools] pwdump6
> > 
> > On 05/18/10 16:42, Keith Morrell wrote:
> > > Hi All...trying to get some more info on pwdump6 (or fgdump).
> > >
> > > I run pwdump6 on a medium Active Directory (~700k entries) and it 
> works fine
> > (after about 20-30 mins) however when I run it on my larger directory 
> of about
> > ~3 million users, it runs for about 3-4 hours and doesn't produce any 
> output.
> > >
> > > Any thoughts?
> > 
> > Those are definitely what I would call massive ADs! We don't run into
> > them all that often, and so we don't do a lot of testing on such 
> sizes.
> > 
> > It will be VERY slow to dump that many users. As luck would have it, I
> > am (right as we speak) working on a new version of pwdump which uses
> > direct registry access to pull in accounts, and should run magnitudes
> > faster. It will probably be a few days yet, but once I get it done, I
> > might be able to get you a beta copy if you want to give that a try.
> > 
> > --f
> > 
> > 
> > _______________________________________________
> > foofus-tools mailing list
> > foofus-tools@lists.foofus.net
> > http://lists.foofus.net/listinfo.cgi/foofus-tools-foofus.net