file - How to organize a large number of objects -

-

we have large number of documents , metadata (xml files) associated these documents. best way organize them?

currently have created directory hierarchy: 

/repository/category/date(when loaded our db)/document_number.pdf , .xml

we use path unique identifier document in our system. having flat structure doesn't seem option. using path id helps keep our data independent our database/application logic, can reload them in case of failure, , documents maintain old ids. yet, introduces limitations. example can't move files once they've been placed in structure, takes work put them way. best practice? how websites such scribd deal problem?

your approach not seem unreasonable, might suffer if more few thousand documents added within single day (file systems tend not cope large numbers of files in directory).

storing .xml document beside .pdf seems bit odd - if it's metadata document, should not in database (which sounds have) can queries , indexed etc?

when storing large numbers of files i've taken file's key (say, url), hashed it, , stored x levels deep in directories based on first characters of hash...

say started key 'how organize large number of objects'. md5 hash 0a74d5fb3da8648126ec106623761ac5 might store at...

base_dir/0/a/7/4/http___stackoverflow.com_questions_2734454_how-to-organize-a-large-number-of-objects

...or can find again given key started with.

this kind of approach has 1 advantage on date 1 in can scaled suit large numbers of documents (even per day) without 1 directory becoming large, on other hand, it's less intuitive having manually find particular file.


Comments

Post a Comment

Popular posts from this blog

javascript - Enclosure Memory Copies -

-
function myclass() { //lots , lots of vars , code here. this.bar = function() { //do numerous enclosed myclass vars } this.foo = function() { alert('hello'); //don't enclosed variables. } } each instance of myclass gets own copy of bar , foo, why prototyped methods use less memory. however, know more memory use of inner methods. it seems obvious me (1) below must true. agree? not distinct myclass instances have own distinct copies of bar, must have own distinct copies of myclass enclosure. (or else how bar method keep myclass variables straight on per instance basis?) now (2) question i'm after. since inner foo method doesn't use in myclass enclosure natural qustion is: javascript smart enough not keep myclass enclosure in memory use of foo? this depend entirely on implementation not language. naive implementations keep far more around others. answer true e.g. v8 (chrome's js engine) may not true spider...
Read more

php - Replacing tags in braces, even nested tags, with regex -

-
example preg_replace('/\{[a-za-z.,\(\)0-9]+\}/', 'replaced', 'lorem ipsum dolor sit {tag1({tag2()})}, consectetur adipiscing elit.'); the result: lorem ipsum dolor sit {tag1(replaced)}, consectetur adipiscing elit. question as can see "tag2" has been replaced, want replace "tag1" know how can this? (in cases might this: {tag1({tag2({tag3()})})}) , on.) btw using preg_replace_callback, easier show preg_replace here site can test code: http://www.spaweditor.com/scripts/regex/index.php you need add curly braces character set. here's pattern used: /\{[a-za-z.,\(\)\{\}0-9]+\}/ and here result: "lorem ipsum dolor sit replaced, consectetur adipiscing elit."
Read more