mongodb - My mongoimport runs to infinity -


i did mongorestore of gzipped mongodump:

mongorestore -v --drop --gzip --db bigdata /volumes/lacie2tb/backup/mongo20170909/bigdata/

but kept going. left it, because figure if 'just' close now, (important) data corrupted. check percentages:

2017-09-10t14:45:58.385+0200    [########################]  bigdata.logs.sets.log  851.8 gb/85.2 gb  (999.4%) 2017-09-10t14:46:01.382+0200    [########################]  bigdata.logs.sets.log  852.1 gb/85.2 gb  (999.7%) 2017-09-10t14:46:04.381+0200    [########################]  bigdata.logs.sets.log  852.4 gb/85.2 gb  (1000.0%)

and keeps going!

note other collections have finished. 1 goes beyond 100%. not understand.

this mongo 3.2.7 on mac osx.

there problem amount of data imported, because there not diskspace.

$ df -h filesystem      size   used  avail capacity   iused     ifree %iused mounted on /dev/disk3     477gi  262gi  214gi    56%  68749708  56193210   55%   /

the amount of disk space used could right, because gzipped backup 200gb. not know if result in same amount of data on wiredtiger database snappy compression.

however, log keeps showing inserts:

2017-09-10t16:20:18.986+0200 command  [conn9] command bigdata.logs.sets.log command: insert { insert: "logs.sets.log", documents: 20, writeconcern: { getlasterror: 1, w: 1 }, ordered: false } ninserted:20 keyupdates:0 writeconflicts:0 numyields:0 reslen:40 locks:{ global: { acquirecount: { r: 19, w: 19 } }, database: { acquirecount: { w: 19 } }, collection: { acquirecount: { w: 19 } } } protocol:op_query 245ms 2017-09-10t16:20:19.930+0200 command  [conn9] command bigdata.logs.sets.log command: insert { insert: "logs.sets.log", documents: 23, writeconcern: { getlasterror: 1, w: 1 }, ordered: false } ninserted:23 keyupdates:0 writeconflicts:0 numyields:0 reslen:40 locks:{ global: { acquirecount: { r: 19, w: 19 } }, database: { acquirecount: { w: 19 } }, collection: { acquirecount: { w: 19 } } } protocol:op_query 190ms

update

disk space still being consumed. 2 hours later, , 30 gb later:

$ df -h filesystem      size   used  avail capacity   iused     ifree %iused  mounted on /dev/disk3     477gi  290gi  186gi    61%  76211558  48731360   61%   /

the question is: there bug in progress indicator, or there kind of loop keeps inserting same documents?

update

it finished.

2017-09-10t19:35:52.268+0200    [########################]  bigdata.logs.sets.log  1604.0 gb/85.2 gb  (1881.8%) 2017-09-10t19:35:52.268+0200    restoring indexes collection bigdata.logs.sets.log metadata 2017-09-10t20:16:51.882+0200    finished restoring bigdata.logs.sets.log (3573548 documents) 2017-09-10t20:16:51.882+0200    done

604.0 gb/85.2 gb (1881.8%)

interesting. :)

it looks similar bug: https://jira.mongodb.org/browse/tools-1579

there seems fix backported 3.5 , 3.4. fix might not backported 3.2. i'm thinking problem might have using gzip and/or snappy compression.


Comments

Post a Comment

Popular posts from this blog

resizing Telegram inline keyboard -

telegram's inline keyboard great feature lots of different use cases. inline buttons added list of items this: inline_keyboard = [[inlinekeyboardbutton(text="button", callback_data="button"), inlinekeyboardbutton(text="reset",callback_data="reset")]] inline_keyboard_markup = inlinekeyboardmarkup(inline_keyboard) update.message.reply_text("hi", reply_markup=inline_keyboard_markup) the above code adds 2 buttons each half width of chat screen. i know the normal keyboard button there resize_keyboard parameter somehow can used. my question is there way resize inline buttons ? example make full width or quarter width. unfortunately, can't now. :( you can suggest @botsupport , might add feature next version.
Read more

command line - How can a Python program background itself? -

consider following python command-line program. doesn't work because incomplete, think illustrates problem: #!/usr/bin/python3 import multiprocessing pool = multiprocessing.pool() task in expensive_iterator_with_user_interaction(): pool.call_async(expensive_computation, (task,)) pool.close() print("dear user, may press ctrl-z , bg script.") pool.join() thus, problem scripts needs user input while it's launching expensive parallel processes. there moment when user interaction not needed anymore. then, ask user background process shell actions. however, can achieved program itself? (side note: core of problem entanglement of user input firing “expensive computations”. therefore, cannot separate both in 2 scripts, , letting 1 spawning other daemon.) the command screen may you. use screen -l create new session , run application. after inputting data, can press " ctrl-a " , press " d " on keyboard detach session. now
Read more

php - "cURL error 28: Resolving timed out" on Wordpress on Azure App Service on Linux -

Image
my wordpress installation on azure app service on linux using custom docker container has slow response times. pages take around 20-40 seconds load. i have troubleshooting plugin installed indicates problem "curl error 28: resolving timed out after n milliseconds" when making requests following urls https://api.wordpress.org/core/version-check/1.7/ http://api.wordpress.org/core/version-check/1.7/ https://api.wordpress.org/plugins/update-check/1.1/ http://api.wordpress.org/plugins/update-check/1.1/ https://api.wordpress.org/themes/update-check/1.1/ http://api.wordpress.org/themes/update-check/1.1/ curl works fine on scm-site command line. example works ok. curl -x post http://api.wordpress.org/core/version-check/1.7/ edit if ssh container , run php code, works fine. <?php $url = 'http://api.wordpress.org/core/version-check/1.7/'; $fields = array( 'version' => urlencode('4.8.1'), 'php' => urlencode(
Read more