Welcome to the Invelos forums. Please read the forum rules before posting.

Read access to our public forums is open to everyone. To post messages, a free registration is required.

If you have an Invelos account, sign in to post.

    Invelos Forums->General: Website Discussion Page: 1... 16 17 18 19 20 ...22  Previous   Next
goodguy's Credit Lookup Plus
Author Message
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
"ReleaseDate" - > "Released" in CLTBoss output is done. Was passing thru that code, so I went ahead and made that change.
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
I am walking up to this post so carefully, so as not to jinx it ...

Implemented custom XPath. I now have total control of the search, using the fastest method.

Since I declared scrape "almost stable" I have had only one scrape error. 99.9% dead on UPC scrapes.

Now, since I "finished" the Xpath code, so far, all dead on match to CLT number. Testing more over the weekend. 

Here is what I had to do to match CLT:

(1) everything case insensitive (I use "ToLower()" on everything, including the entire source xml)
(2)  squeeze multiple blanks from search field and trim leading and trailing blanks
(3) do the same for all target data fields (first, middle, last, creditedas)
(4) concatenate (F+" "+M+" "+L)
(5) squeeze and trim the concatenated result
(6) Use (target field).Contains(search field), NOT (target == search) to capture stuff like "queen elizabeth II"

That's it so far. If my luck holds with testing, I will start GUI cleanup, code cleanup, and revisit my "to do" list, and try to publish a video over the weekend. Hopefully release code early next week. Cross fingers ... 

Edit: I will be testing some big guns: Christopher Lee, Zhang Ziyi, Tom Cruise, Clint Eastwood and others for variety and complexity and size, and do timings as well, to see if I am getting close to CLTPlus speed.
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
 Last edited: by mediadogg
DVD Profiler Unlimited RegistrantStar ContributorGSyren
Profiling since 2001
Registered: March 14, 2007
Reputation: Highest Rating
Sweden Posts: 3,753
Posted:
PM this userVisit this user's homepageView this user's DVD collectionDirect link to this postReply with quote
Sounds like you're close to the finish line (knock on wood). It's been a long and ardous journey. I'm glad that you stuck it out. 

Really looking forward to running some test in order to tweak CLTinfo.
My freeware tools for DVD Profiler users.
Gunnar
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
Quoting GSyren:
Quote:
Sounds like you're close to the finish line (knock on wood). It's been a long and ardous journey. I'm glad that you stuck it out. 

Really looking forward to running some test in order to tweak CLTinfo.

Thanks. It is good to have someone in the boat, rowing with you against the tide.

Christopher Lee 3504 spot on scrape in 9 min. But only 3460 profile hits in 47 min.

So, I will be spending time today, after breakfast, chores, email and tutoring sessions ( ), figuring out which profiles were missed and why, and maybe starting on some code cleanup and performance tweaking.

Then again, my wife bought me a new battery operated miter saw. I might go do a couple of woodworking projects since it cooled off a bit today. 

Do you want a link to the Christopher Lee Invelos XML? It is big, but it will be a link to my drop box. 62MB with 3460 profiles in Invelos format (I think. Nobody but CookTop will load it, and it doesn't have a count). This is pure Invelos format (plus variants), so there shouldn't be any wasted effort in testing with it.

I have implemented ways to reuse and "repair" incomplete UPC scans. I also need to add a similar way to insert XML, so you can run UPCs you missed, or add new ones without spending an hour running those you already have.
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
 Last edited: by mediadogg
DVD Profiler Unlimited RegistrantStar ContributorGSyren
Profiling since 2001
Registered: March 14, 2007
Reputation: Highest Rating
Sweden Posts: 3,753
Posted:
PM this userVisit this user's homepageView this user's DVD collectionDirect link to this postReply with quote
Quoting mediadogg:
Quote:
Do you want a link to the Christopher Lee Invelos XML?

Thanks, but that will not be neccessary. I'm now more interested in variety that volume.
I took my collection export file and added two variant nodes to it. 8800 profiles loaded and analyzed in 13 seconds. Given that even the worst cases, like Christopher Lee, doesn't come to half that, I think I don't need to spend my time trying to speed that up.

I'm interested in files where there is a mix of F/M/L and Credited As hits, to make sure I count the credits correctly. But that can wait until you have a version that you feel you can share, so I can experiment myself. My bit is a walk in the park, compared to the complexity of CLTBoss, so I don't want to burden you with stuff I can do myself.
My freeware tools for DVD Profiler users.
Gunnar
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
What a delightfully considerate response, and I really appreciate it.
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
Danny Devito kicking my butt big time. Perfect scrape of 2560 UPCs from CLT. However my credits scan picks up only 2020 profiles!!!!!!!!!!! Why is it that I get perfect results for actors with hundreds of profile credits, and get errors on guys with thousands????       

Makes it harder to debug. Oh well, enough whining ... back to work.

Oh wait, what if I scan in batches of 100. Could it be a memory issue? Hmmm ...
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
Found a bug that nearly resolves it. Still a mysterious gap that somehow exactly equals the CreditedAs matches. But they should be in the count already, so there must be counting bug somewhere.
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
The good news:
- beginning to see stability. Get same results for same actors, every time

The bad news:
- there is a subtle problem that causes me to consistently get the wrong answer for certain actors

I have a set of actors that I use for testing that span from butt easy as a quick check, to difficult due to size, complexity of credits, corrupt profiles, etc. I constantly re-run the same ones to make sure I haven't broken something or to see if I have fixed the known errors.

For example, these all run totally 100% correct each time:

- queen elizabeth
- donald trump
- will wallace
- and many others

But for reasons I am still figuring out, I get consistently the same shortage of profile hits for:

- danny devito
- tom cruise

Cross fingers, I have been getting 100% scrapes for some time now. But there is something with the XML Xpath scan. I am just talking out loud to blow off some frustration. I am not actually asking for any specific help at the moment. I suspect that I have error trapping code that is hiding an error of some sort that causes the skipping of certain profiles. Could be an Xpath error, unexpected characters in the datastream, out of memory error, something like that ... (and oh yes, I have an error log.)

It wouldn't hurt to have CLTPlus output for Danny Devito, if available. It would be gravy - I probably have what to need to solve it without that additional data,
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
 Last edited: by mediadogg
DVD Profiler Unlimited RegistrantStar ContributorGSyren
Profiling since 2001
Registered: March 14, 2007
Reputation: Highest Rating
Sweden Posts: 3,753
Posted:
PM this userVisit this user's homepageView this user's DVD collectionDirect link to this postReply with quote
Quote:
It wouldn't hurt to have CLTPlus output for Danny Devito, if available.

I'd like to help, but I can no longer get CLTPlus to work, even though I am logged in to Invelos in Internet Explorer. 
My freeware tools for DVD Profiler users.
Gunnar
DVD Profiler Desktop and Mobile RegistrantStar ContributorAiAustria
Profiling since 2004
Registered: May 19, 2007
Reputation: Highest Rating
Austria Posts: 5,172
Posted:
PM this userDirect link to this postReply with quote
Complete list of Common Names  •  A good point for starting with Headshots
DVD Profiler Desktop and Mobile RegistrantStar ContributorAiAustria
Profiling since 2004
Registered: May 19, 2007
Reputation: Highest Rating
Austria Posts: 5,172
Posted:
PM this userDirect link to this postReply with quote
Quoting GSyren:
Quote:
Quote:
It wouldn't hurt to have CLTPlus output for Danny Devito, if available.

I'd like to help, but I can no longer get CLTPlus to work, even though I am logged in to Invelos in Internet Explorer. 

Are you aware of the fact, that you have to log in with administrative rights?

Edit: ... and you have to type the www. in front of the domain: http://www.dvdprofiler.com
Complete list of Common Names  •  A good point for starting with Headshots
 Last edited: by AiAustria
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
Thanks to my fellow soldiers. Hopefully I will not have to request this again. With the ones I have now, and the debugging tools I have been gradually accumulating, I should be able to figure out what is going on.

To give you a flavor, yesterday, I was suddenly getting a weird error when I was loading and saving past results in CSV format. Turns out the title of some movie in some language I don't know, had a ";" in the title as part of the way that character set is represented in non-unicode. But that character is deemed to be a  delimiter, so I had to modify my error trapping to let that one pass, in this circumstance.

Edit: FYI. "Páginas de Liberdade (Freedom Writers)"  (holy smokes did you see that? got translated to A tilde i - but when loading the raw data, I assure you there was a ; in there - WTF?" (CSV processor shares my bad eyes. The character is not ";" but also not "i" on the same code page.)
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
 Last edited: by mediadogg
DVD Profiler Desktop and Mobile Registrantmediadogg
Aim high. Ride the wind.
Registered: March 18, 2007
Reputation: Highest Rating
United States Posts: 5,556
Posted:
PM this userVisit this user's homepageDirect link to this postReply with quote
Aha, so even CLTPlus does not always 100% match CLT! I didn't get any parsing errors when loading the file, and it has only 2559 profiles, not 2560 as CLT reports. Anyway, still better than 2539. 

Hang on maybe spoke too soon ... doing a quick check, why I am missing one on load ...

Edit: Yeah, there are actually 2560 profile entries. I must have spat one out for some reason. 
Thanks for your support.
Free Plugins available here.
Advanced plugins available here.
Hey, new product!!! BDPFrog.
 Last edited: by mediadogg
DVD Profiler Unlimited RegistrantStar ContributorGSyren
Profiling since 2001
Registered: March 14, 2007
Reputation: Highest Rating
Sweden Posts: 3,753
Posted:
PM this userVisit this user's homepageView this user's DVD collectionDirect link to this postReply with quote
Quoting AiAustria:
Quote:
you have to type the www. in front of the domain: http://www.dvdprofiler.com

Ah, maybe that's where I went wrong. Thanks!
My freeware tools for DVD Profiler users.
Gunnar
DVD Profiler Unlimited RegistrantStar ContributorGSyren
Profiling since 2001
Registered: March 14, 2007
Reputation: Highest Rating
Sweden Posts: 3,753
Posted:
PM this userVisit this user's homepageView this user's DVD collectionDirect link to this postReply with quote
Quoting mediadogg:
Quote:
Aha, so even CLTPlus does not always 100% match CLT! I didn't get any parsing errors when loading the file, and it has only 2559 profiles, not 2560 as CLT reports. Anyway, still better than 2539. 

Hang on maybe spoke too soon ... doing a quick check, why I am missing one on load ...

You should go through all entries in the CLT manually, and see if one of them maybe has an error!   
My freeware tools for DVD Profiler users.
Gunnar
    Invelos Forums->General: Website Discussion Page: 1... 16 17 18 19 20 ...22  Previous   Next