Android - Save big JSON to Realm OutOfMemoryException -


situation:

i have large json file on server ( >= 50-100mb ) information of customers need on app.

i have download file application , save information realm database. (there 200k objects in file).

customer.class

this object saved in array:

public class customer  extends realmobject {     public customer(){}      @primarykey     private string id;     private string full_name;     private string phone_no;     private string codice_fiscale;     private date modified_on;      //getters , setters

my download , save code:

this asynctask, removed useless code easier view

    @override     protected void doinbackground(string... params) {         realm realm = null;         newsha = null;         try {             realm = realm.getdefaultinstance();              string url = "myurl";             okhttpclient client = new okhttpclient.builder()                     .connecttimeout(10, timeunit.seconds)                     .writetimeout(120, timeunit.seconds)                     .readtimeout(120, timeunit.seconds)                     .build();              request request = new request.builder().url(url)                     .addheader("content-type", "application/json")                     .build();              response response = client.newcall(request).execute();             responsebody responsebody = response.body();              try {                 string myheader = response.header("content-disposition");                 if (myheader != null) {                     int position = myheader.indexof("filename=");                     position += "filename=".length();                     newsha = myheader.substring(position);                 }             } catch (exception ex) {                 newsha = null;             }              if (responsebody == null) {                 return null;             }             inputstream = responsebody.bytestream();             bufferedreader reader = new bufferedreader(new inputstreamreader(is));             final stringbuilder sb = new stringbuilder();              string line;             while ((line = reader.readline()) != null) {                 sb.append(line);             }              response.close();             responsebody.close();             is.close();             reader.close();              //switch because have 20-30 cases, big file gives problems             switch (filename) {                 case "customers": {                     realm.executetransaction(new realm.transaction() {                         @override                         public void execute(realm realm) {                             try {                                 log.wtf("sync", "customer - start delete");                                 realm.delete(customer.class);                                 log.wtf("sync", "customer - end delete, start create");  //this line reached                                 realm.createallfromjson(customer.class, sb.tostring());                                   log.wtf("sync", "customer - end create");                //this line never reached                                 settitle(string.format("completato! [%s]", title));                                  publishprogress(100, 1);                                 addsynccompleted(false, null, null);                                 genericdbclass.setsha(realm, new customer(), newsha);                             } catch (exception error) {                                 newsha = null;                                            //this error never thrown                             }                         }                     });                 }                 break;

problem:

the realm.createallfromjson throws outofmemoryexception (not thrown, application shut down no catch fired , without error. can see in log exceptions logged warn

w/art: throwing outofmemoryerror "failed allocate 170348044 byte allocation 16777216 free bytes , 162mb until oom"

the problem on how save big json file realm. tried parse string earlier gson without success. file might increase size in time, how can speed up?

any appreciated, i've been stuck on 1 week.

realm has streaming variant of createallfromjson() accepts inputstream directly: https://realm.io/docs/java/3.7.1/api/io/realm/realm.html#createallfromjson-java.lang.class-java.io.inputstream-

this should prevent running out of memory.


Comments

Post a Comment

Popular posts from this blog

Sort a complex associative array in PHP -

this question has answer here: php sort array subarray value 5 answers i have associative array in php like: $newarray1 = array ( [producta] => array ( [link] => http://sanjosespartan.com/blog/products/agile-java-development/ [visitcount] => 4 ) [productd] => array ( [link] => http://sanjosespartan.com/blog/products/intro-to-go/ [visitcount] => 1 ) [productg] => array ( [link] => http://sanjosespartan.com/blog/products/node-js-for-developers/ [visitcount] => 1 ) [productb] => array ( [link] => http://sanjosespartan.com/blog/products/beginning-mysql/ [visitcount] => 1 ) [productc] => array ( [link] => http://sanjosespartan.com/blog/products/computer-networks/ [visitcount] => 1 ) ...
Read more

vb.net - How to ignore if a cell is empty nothing -

how use roscolor() ignore value (like empty cell) ? color row if value upper 5 when there nothing in cell, want ignore roscolor() apply, how? public sub roscolor() integer = 0 quotedatagridview1.rows.count() - 1 step +1 dim val integer val = quotedatagridview1.rows(i).cells(3).value if val = vbempty quotedatagridview1.rows(i).defaultcellstyle.backcolor = color.white elseif val < 5 quotedatagridview1.rows(i).defaultcellstyle.backcolor = color.red elseif val > 5 , val < 10 quotedatagridview1.rows(i).defaultcellstyle.backcolor = color.lightyellow end if next end sub you can check empty value following public sub roscolor() integer = 0 quotedatagridview1.rows.count() - 1 step +1 dim val = quotedatagridview1.rows(i).cells(3).value if isdbnull(val) or val nothing quotedatagridview1.rows(i).defaultcellstyle.backcolor = color.white ...
Read more

recursion - Can every recursive algorithm be improved with dynamic programming? -

i first year undergraduate csc student looking competitive programming. recursion involves defining , solving sub problems. understand, top down dynamic programming (dp) involves memoizing solutions sub problems reduce time complexity of algorithm. can top down dp used improve efficiency of every recursive algorithm overlapping sub problems? dp fail work , how can identify this? the short answer is: yes. however, there constraints. obvious 1 recursive calls must overlap. i.e. during execution of algorithm, recursive function must called multiple times same parameters. lets truncate recursion tree memoization. can use memoization reduce number of calls. however, reduction of calls comes price. need store results somewhere. next obvious constraint need have enough memory. comes not-so obvious constraint. memory access requires time. first need find result stored , maybe copy location. in cases, might faster let recursion calculate result instead of loading somewher...
Read more