{"id":438,"date":"2023-02-06T22:32:04","date_gmt":"2023-02-06T21:32:04","guid":{"rendered":"https:\/\/Pallieter.org\/www8\/?p=438"},"modified":"2023-02-06T22:32:04","modified_gmt":"2023-02-06T21:32:04","slug":"google-drive-sqlite-reference","status":"publish","type":"post","link":"https:\/\/Pallieter.org\/www8\/2023\/02\/06\/google-drive-sqlite-reference\/","title":{"rendered":"Google Drive SQLite Reference"},"content":{"rendered":"\n<p>As a reference, here are the databases Google uses (as per 20230206), as I could not find this anywhere on the internets. Reason I was diving deeper into this: I have a 4+ million files GDrive (0.6TB) and a resync (data already local) on a new machine took several months to complete while running the machine at ~7% CPU with 8GB mem usage. I wanted to see the synchronization progress as the GUI did not provide it.<\/p>\n\n\n\n<p>Files:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>experiments.db<\/li>\n\n\n\n<li>metrics_store_sqlite.db (from root folder)<\/li>\n\n\n\n<li>root_preference_sqlite.db<\/li>\n\n\n\n<li>chunks.db<\/li>\n\n\n\n<li>metrics_store_sqlite.db (from the data folder)<\/li>\n\n\n\n<li>mirror_metadata_sqlite.db (this is the BIG and juicy one)<\/li>\n\n\n\n<li>mirror_sqlite.db (the current workload)<\/li>\n<\/ul>\n\n\n\n<p>Some other keywords:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>C:\\Users\\UserName\\AppData\\Local\\Google\\DriveFS\\<\/li>\n\n\n\n<li>\/depot\/branches\/drive_fs_release_branch\/0.0\/google3<\/li>\n\n\n\n<li>Mirror root context menu for multiple items is not supported.<\/li>\n\n\n\n<li>Passthrough is not supported, GL is disabled, ANGLE is<\/li>\n\n\n\n<li>Crashpad: The system cannot find the file specified.<\/li>\n\n\n\n<li>C:\\Program Files\\Google\\Drive File Stream\\0.0.0.0\\GoogleDriveFS.exe<\/li>\n\n\n\n<li>C:\\Users\\UserName\\AppData\\Local\\Google\\DriveFS\\Logs\\drive_fs.txt<\/li>\n<\/ul>\n\n\n\n<p>The mirror_metadata_sqlite.db file contained the table stable_ids that lists files that look ready (a rowcount indicated the work that was done) and via PowerShell I did a total file count on my GDrive folder to calculate the difference.<\/p>\n\n\n\n<p>mirror_sqlite.db calculates hashes (local_md5_checksum, cloud_md5_checksum) on both the could and local to see if files need updating. It also tracks file names (local_filename, cloud_filename), filetime (local_mtime_ms, cloud_mtime_ms), etc.<\/p>\n\n\n\n<p>SELECT COUNT(stable_id) FROM stable_ids;<\/p>\n\n\n\n<p>PS: The &#8220;why is this important&#8221; question: because it will not sync newer versions of files until it processed the rest, so that means manual updating local files with the cloud files. Note: this causes new file versions, so requires extra storage (I have not yet found a way to prevent this).<\/p>\n\n\n\n<p>PPS: Google Support is really really bad &#8211; they made me jump through stupid hoops on repeat with no answers or results. See also <a href=\"https:\/\/www.googlenestcommunity.com\/t5\/Nest-Wifi\/Google-Nest-Wifi-Router-unable-to-acquire-IPv6-How-to-set-up-prefix\/m-p\/261044\/highlight\/true#M28617\">this link<\/a> as to why you should not buy their WiFi hardware and if you buy a Nest Security Camera, they promise replacement cameras if they get stolen, but after weeks of emailing, the manager decided: no. So, use <a href=\"http:\/\/www.backblaze.com\/partner\/af0240\">BackBlaze<\/a> and Ubiquity!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>As a reference, here are the databases Google uses (as per 20230206), as I could not find this anywhere on the internets. Reason I was diving deeper into this: I have a 4+ million files GDrive (0.6TB) and a resync (data already local) on a new machine took several months to complete while running the [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-438","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/posts\/438","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/comments?post=438"}],"version-history":[{"count":1,"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/posts\/438\/revisions"}],"predecessor-version":[{"id":439,"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/posts\/438\/revisions\/439"}],"wp:attachment":[{"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/media?parent=438"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/categories?post=438"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/Pallieter.org\/www8\/wp-json\/wp\/v2\/tags?post=438"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}