| Message |
Details |
This was an interesting challenge. Unfortunately it turned out that the data does not contain all cast and crew. Not even close.
So what the point of providing these files? Most disappointing. 
Oh, well. I got to play a little with SQLite. Maybe this experience will be useful some time. | Posted: Topic Replies: 13, Topic Views: 440 |
 |
I didn't think that it could be sped up enough to make it palatable, but how wrong I was! 
I changed the program to use transactions of 10,000 records each. Now it took 36 minutes instead of (the estimated) 5 days! Together with the other two files I needed, building the database took about an hour (not counting the coding time).
Now I just have to build a program that queries that database. But that's a job for another day.
Thanks for pushing me, ObiKen! 
PS The database became almost 7 GB. | Posted: Topic Replies: 13, Topic Views: 440 |
|
I tested loading the largest file into an Sqlite table. I found that I could load about 200 line per second. That would mean that it would take over 5 days (!) to complete.
I'm a fairly patient guy, but I would go bananas  waiting that long, so I aborted the test.
It's nice of IMDb to dump their database tables into downloadable files (for non-commercial use), but I can't help but wonder who actually finds a use for them.
So, this whole project was pretty much a waste of time. Oh, well, I did get a little refresher on how to use Sqlite, so at least that's something. I wonder if I can think of something else where I can use it ...  | Posted: Topic Replies: 13, Topic Views: 440 |
|
Quoting ObiKen:
Quote:
My mantra is the solution has got to be simple, when I ask for a banana, I don't want a gorilla holding that banana.
I take it that you didn’t realize you were suggesting a gorilla?
I think I may build this for myself, to see if it climbs the Empire State Building with the banana.
"T’was fruit killed the beast"  | Posted: Topic Replies: 13, Topic Views: 440 |
|
Just for some perspective, I ran a line count on all the IMDb files:
title.principals.tsv - 99 738 115 lines
title.akas.tsv - 57 439 274 lines
name.basics.tsv - 15 378 039 lines
title.basics.tsv - 12 539 092 lines
title.crew.tsv - 12 537 727 lines
title.episode.tsv - 9 685 416 lines
title.ratings.tsv - 1 676 413 lines
I guess it would be theoretically possible to write a program that loads (some of) these files into a database, and get full cast and crew listings from it. title.principals.tsv and title.basics.tsv would probably suffice for movies, add title.episode.tsv for TV shows. But would anyone be interested? | Posted: Topic Replies: 13, Topic Views: 440 |
|
Quoting ObiKen:
Quote:
What about going direct to the IMDb Non-Commercial Datasets here.
Yeah, I looked at that. 7 files totaling over 9 GB unzipped. 6.6 GB if you skip AKAS and Ratings. Still you would need to load those files into some database in order to be able to extract any meaningful information out of them. Trying to use those flat files directly would mean that any lookup would take forever. And if you wanted to keep to it up to date, you'd have to go through the whole download/unzip/load again. And again ... 
So not really a useful option, I'm afraid. | Posted: Topic Replies: 13, Topic Views: 440 |
|
That may be true, but that’s beside the point. The data should be checked against the credits anyway. You use it as a starting point. However, if the data is not complete, you’re no better off than if you use data from TMDb.
Also, transforming crew jobs to Profiler format is hopeless if the source isn’t consistent.
I may take another look at this in the future. But it feels like too little, too late. I’m not sure that there are enough users who care, in order for this to be worth the effort. | Posted: Topic Replies: 13, Topic Views: 440 |
|
Inspired by a post in the Cast/Crew Edit 2 thread, I started to look for AI APIs that would allow me too look up IMDb entries. ChatGPT said it could, but the API wouldn't be free. Well, for a freeware program that is a no-no, in my opinion. So I turned to Microsoft Copilot. It said that it couldn't provide IMDb data via the API. I didn't bother to check if the API was free. Finally I asked Gemini if it could do it. It said that it could, and that there was a free API (with some limitations).
That sounded promising, so I did some initial testing, and it worked. I instructed Gemini that I wanted all cast and crew, and I wanted the credits exactly as in IMDb. On closer testing I found that Gemini failed in both aspects. Running the same movie multiple times would produce different number of credits, and slight variation of roles and crew jobs - not as credited. I tried changing the prompt to impress the criteria on Gemini, but that didn't help. That isn't good enough.
So, at least for now, this idea has been shot down. Disappointing, because I was really hoping to be able to mine IMDb data without having to resort to screen scraping. So I guess it's still either Cast/Crew Edit 2, or being content with TMDB data using TmdbInfo.  | Posted: Topic Replies: 13, Topic Views: 440 |
|
GsyLaunch 2.8.1
The most annoying things for me just after a program has been released are if I find a bug, or if I realize that I have overlooked something obvious that would have made the program better. In this case it is the latter.
The difference between an ordinary shortcut and a UWP shortcut is that GsyPlayer still needs the UWP shortcut. You can't just delete it. But you can move it away from the desktop. And that's what I initially suggested that the user would do. But for some reason I overlooked the (in hindsight blindingly obvious) solution to let GsyPlayer copy the entire shortcut to some suitable location and use it from there.
So in GsyPlayer 2.8.1, if you drop a UWP shortcut into it, the program copies the shortcut into its appdata folder and uses it from there. Thus you can delete the original shortcut and keep your desktop as clear as you like. So for the user it works just the same as dropping a regular shortcut (or an exe file).
In addition I have made two cosmetic changes. I don't know how to find out what the process name of a certain UWP program will be, so I cannot tell if it is running or not. So for UWP programs, the "Prevent double run" switch in the properties will be disabled. And the tooltip for the button will always have a grey background.
And as usual, if you have recently visited the web page, you may have to refresh your browser to see the latest change. | Posted: Topic Replies: 27, Topic Views: 33365 |
|
Thanks woodg! I overlooked changing the web page. I will fix that.
And I'm glad you find GsyLaunch useful. That makes at least two of us.  | Posted: Topic Replies: 27, Topic Views: 33365 |
|
GsyLaunch 2.8.0
It's been some time since I did anything about GsyLaunch. The one thing that annoyed me was that I couldn't figure out how to add shortcuts from UWP programs (such as programs from Windows Store). Well, I finally figured out how to do that.
There are a couple of things that you need to know, though. I have documented them in the help file. And there is now a menu item for the help file, so you don't have to browse through the files to open it. | Posted: Topic Replies: 27, Topic Views: 33365 |
|
Quoting ObiKen:
Quote:
When you start DVDP, a "Temp" folder is created in the username/AppData/Local/DVD Profiler/ folder on your Windows PC.
After opening the initial profile in DVDP, the program creates two temp files (with .DAT and .IDX extensions) in this Temp folder.
I do see those two files there, but they are always there, not created when I start editing. They do seem to get modified when I edit something though. | Posted: Topic Replies: 6, Topic Views: 590 |
|
| I think what you’re seeing is a PM block. That shouldn’t prevent you from seeing their contributions. | Posted: Topic Replies: 14, Topic Views: 987 |
|
I agree. And what makes it even more scary is that this person has 1660 accepted profile contributions. One has to wonder how much crap that they have gotten accepted into the online database ...  | Posted: Topic Replies: 14, Topic Views: 987 |
|
| Odd! I don't see that happening. There must be something else that is interfering. | Posted: Topic Replies: 6, Topic Views: 590 |
|
Sounds great!
Just a thought for Magmadrag: since the only optional data that you need in the export probably is Tags, you can uncheck all the rest. I did a quick test, and it reduced my export file by almost 70%. | Posted: Topic Replies: 17, Topic Views: 1695 |
|
Quoting mediadogg:
Quote:
I'll bet if you dump your xml file into Claude, you could analyze anything within seconds.
That's a thought, but I wonder if the free version of Claude can handle his xml file? He has over 34,000 profiles so the exported file is going to be huge. Worth a try, though. | Posted: Topic Replies: 17, Topic Views: 1695 |
|
| Not sure what Magmadrag's issue with tags is. Tags are available in ProfilerQuery (provided that you include them in the xml export, of course). | Posted: Topic Replies: 17, Topic Views: 1695 |
|
Sounds complicated 
Still, I'm glad that you found ProfilerQuery useful. Best of luck! | Posted: Topic Replies: 17, Topic Views: 1695 |
|
I'm not sure I quite understand your problem, but maybe this can be helpful?
If you (supposedly) set count to zero on all child profiles, you can add a filter on child or parent to make sure that
a) All child profiles really have count as = 0
b) All non-child profiles have count = 1 | Posted: Topic Replies: 17, Topic Views: 1695 |
|
Quoting T!M:
Quote:
If I'm going to "IMDb to DVD Profiler Transformation Data" in the "Edit Config File" menu, it immediately throws an error.
Same here, but I get this:
 | Posted: Topic Replies: 776, Topic Views: 71902 |
|
Just FYI, in ProfilerQuery you can list Count As, and you can also filter on Count As, so you can get a list of profiles where it is not set to one.
 | Posted: Topic Replies: 17, Topic Views: 1695 |
|
I tested the latest version, and got no errors on scanning, but also got no data.
So, just a reminder, if CastCrewInfo isn't working you can get cast and crew with TmdbInfo.
It's not quite as advanced, but it is better than nothing (and it doesn't need constant updates since the Tmdb API doesn't change). | Posted: Topic Replies: 776, Topic Views: 71902 |
|
I initially voted based on the screenshot, but after revisiting the thread I have changed my mind.
So now it’s even less definitive at 13 to 10. | Posted: Topic Replies: 7, Topic Views: 1612 |
|
TmdbInfo 3.1.1
If you used TmdbInfo together with my tool CastCrewCheck, selecting Copy Cast or Copy Crew could cause en exception.
Fixed in this version. | Posted: Topic Replies: 7, Topic Views: 2285 |