Forgot Password
Pentax Camera Forums Home
 

Reply
Show Printable Version Search this Thread
11-10-2018, 01:52 PM   #1
Digitiser of Film
Loyal Site Supporter
BigMackCam's Avatar

Join Date: Mar 2010
Location: North East of England
Posts: 20,575
Setting up HDD for reliable read/write access from both Windows 10 and Ubuntu / Mint?

Apologies, I realise this isn't strictly about digital processing and software, but it's related and I hope I'll get more visibility for my request by posting in this forum

Having originally been a Windows user, then migrated to Linux Mint 18.3, soon I will be setting up a machine with Windows 10 and Linux (either Ubuntu or Mint) as dual boot.

Both of the operating systems (and associated applications software) will be installed on the machine's first drive, a 512GB PCIe NVMe SSD, partitioned approximately 50/50. Also in the machine will be a 2TB HDD (or SSHD) for file storage. If possible, I would like to set up this drive to be reliably accessible (read and write) by both Windows 10 and Linux.

For the distros I'm looking at, I believe I can share directories in my Windows file system for use in Linux, but I'd rather not to have permissions for this controlled from, or limited by, Windows. I'd prefer to have that HDD independently accessible from each operating system without relying on anything in the other.

From the reading up I've done so far, it seems to me that the most established method would be to format the HDD as NTFS, since this is still a native format for Windows and - as I understand it - NTFS support has been pretty fundamental to Linux for some time (certainly for Ubuntu and variants).

Does this make sense, and do any of our members make use of this approach? If so, are there any problems or gotchas I should be aware of?

Alternatively, is there a better approach?

Thanks in advance!

11-10-2018, 02:00 PM   #2
Site Supporter
Site Supporter
microlight's Avatar

Join Date: Sep 2011
Location: Hampshire, UK
Posts: 2,127
Woooo! Sounds complicated to me, Mike - two pooters? 😜
11-10-2018, 02:26 PM - 1 Like   #3
Senior Member




Join Date: Oct 2018
Posts: 242
I trust ext4 more in holding data then NTFS or FAT32, especially in case of file system errors and data recovery. I use network drives via samba/nfs, the only method to have data accessible across all Linux, WIn, Mac, Android, etc. Linux's NTFS support should be mature enough, but NTFS itself is lacking features.
11-10-2018, 02:33 PM   #4
Digitiser of Film
Loyal Site Supporter
BigMackCam's Avatar

Join Date: Mar 2010
Location: North East of England
Posts: 20,575
Original Poster
QuoteOriginally posted by microlight Quote
Woooo! Sounds complicated to me, Mike - two pooters? ��
Ha ha No, just the one...

The all-singing-all-dancing gaming laptop I recently bought (to replace my 5 - 6 year old HP ENVY 17) had to go back to the manufacturer twice within less than three weeks from delivery due to different problems, and I'm now getting a refund

Although it's difficult to justify the expenditure, I've decided to splash out and get a machine that's built for longevity, so I'm looking at one of HP's mobile workstations - either the "budget" 15v G5 with Quadro P600 and 15.6" 100% sRGB 4k 340 nits screen, or the pricier but heavier-duty 15 G5 with P1000 and 15.6" 100% AdobeRGB 4k 600 nits screen. I'm still debating which... My pocket firmly says the former, but the latter has an aluminium chassis with no flex to base or screen, more room for growth and expansion should I ever need it, and the option to use AdobeRGB colour space effectively if required - though I somehow doubt that's ever going to be an issue for me, and the wider gamut screen might actually be a hindrance if I don't actually need it, as I'd have to swap between profiles depending on workflow.


Last edited by BigMackCam; 11-10-2018 at 02:53 PM.
11-10-2018, 02:36 PM   #5
Digitiser of Film
Loyal Site Supporter
BigMackCam's Avatar

Join Date: Mar 2010
Location: North East of England
Posts: 20,575
Original Poster
QuoteOriginally posted by K1N8 Quote
I trust ext4 more in holding data then NTFS or FAT32, especially in case of file system errors and data recovery. I use network drives via samba/nfs, the only method to have data accessible across all Linux, WIn, Mac, Android, etc. Linux's NTFS support should be mature enough, but NTFS itself is lacking features.
Thanks!

Interesting... So does Win10 have native support for read/write of ext4? I was under the (probably misguided) impression that 3rd party software was required?

I should also say that I'm pretty disciplined in backing up my data to external drives... Though I'd clearly prefer not to lose it in the first place
11-10-2018, 02:55 PM - 1 Like   #6
Site Supporter
Site Supporter
lsimpkins's Avatar

Join Date: Feb 2013
Location: SE Pennsylvania
Photos: Gallery | Albums
Posts: 537
QuoteOriginally posted by BigMackCam Quote
Thanks!

Interesting... So does Win10 have native support for read/write of ext4? I was under the (probably misguided) impression that 3rd party software was required?

I should also say that I'm pretty disciplined in backing up my data to external drives... Though I'd clearly prefer not to lose it in the first place
Found this chart on the How to Geek website. Hope it helps.
Attached Images
 
11-10-2018, 03:11 PM - 1 Like   #7
Site Supporter
Site Supporter




Join Date: Mar 2008
Location: Prince George, BC
Photos: Gallery | Albums
Posts: 3,543
Hi, Mike. Firstly, I am sort of glad that you had to send your Chinese laptop back. It was giving you problems that should not have been there.

As for your file access question, there are usually two routes to access Windows files from linux: Samba/NTFS or NFS. Both methods have been mature for quite a while on the linux side and each have their advantages. Best you google for setups for these to see what fits your needs. Of course, we are relying on MS to not throw a wrench into their code somewhere but they have been cozying up to Ubuntu in particular the past few years so I doubt they would do this. Some say MS will eventually make a play to buy out Ubuntu's parent company Canonical in the not too distant future but that is speculation. Shuttleworth still has enough big bucks to fend off MS.

Jack

11-10-2018, 03:17 PM   #8
Senior Member




Join Date: Oct 2018
Posts: 242
QuoteOriginally posted by BigMackCam Quote
Interesting... So does Win10 have native support for read/write of ext4? I was under the (probably misguided) impression that 3rd party software was required?
Sorry, I don't think there is any file system that can "reliably" be used across all operating systems, that supports modern features like journaling :/

I am thinking 10 years ahead here. That ext4 driver for win10 might break with a Windows update. FAT32 works everywhere, but it has too many restrictions, maximal file and disk size and things like that.

There is no simple choice that works for all :/
11-10-2018, 03:17 PM - 1 Like   #9
Pentaxian
Dartmoor Dave's Avatar

Join Date: Aug 2012
Location: Dartmoor, UK
Photos: Gallery
Posts: 3,857
I've been dual booting Windows 7 and Ubuntu for a few years with the shared drive formatted to NTFS, and both operating systems have handled it okay. In fact, I've actually had more problems dual booting Windows 7 and Windows 10 causing a frequent need to run CHKDSK.

The main thing to watch out for is that Linux allows longer file names than Windows, which can be hassle. You have to then extract the 8 character file name from the Windows command line, and I once lost a file when that failed.
11-10-2018, 03:20 PM   #10
Digitiser of Film
Loyal Site Supporter
BigMackCam's Avatar

Join Date: Mar 2010
Location: North East of England
Posts: 20,575
Original Poster
QuoteOriginally posted by lsimpkins Quote
Found this chart on the How to Geek website. Hope it helps.
Thanks, Lee I hadn't come across that, so it's very helpful in addition to any other feedback I get in this thread.

FAT32 clearly looks like the winner, though I worry (perhaps unnecessarily?) about the maximum file size of 4GB. That's a BIG file, but I'm thinking of situations - such as creating compressed archives - that might potentially break that limit. Next in the best-supported list, it would seem, is NTFS - which ties in with what I'd read. So I guess until I learn of reasons not to go with NTFS, that remains my default assumption for best choice...
11-10-2018, 03:42 PM - 1 Like   #11
Pentaxian
photoptimist's Avatar

Join Date: Jul 2016
Photos: Albums
Posts: 5,113
IANAWU and IANALU (not a windows or linux user) but I've suffered through enough OS, storage, and filesystem nonsense to know where a few gotchas might lie.

First, I'm assuming and hoping that each OS accesses this disk while the other OS is completely shut down. That is, each OS gets to reliably unmount the HDD before the other OS is booted and mounts the HDD. If not, you might need some clever intermediate something to share the HDD.

Second, after you load your data the first time, run whatever disk/file system checking utilities you like on both OSes. Pay attention to any complaints/repairs required by one OS and then rerun the utility of the other OS to make sure they are going to fight over the file system structure.

Third, watch out for any OS updates or upgrades that "update" the file system or add features to it. These may break compatibility with the other OS.

Good luck and enjoy your new dual system!
11-10-2018, 03:48 PM - 1 Like   #12
Site Supporter
Site Supporter




Join Date: Jun 2008
Location: Idaho
Photos: Gallery
Posts: 2,360
The issues I've had with FAT32 deal with recording video and pose a size (time) limit on how much can be continuously recorded. Some video authoring apps can be limited or excluded by that limit. It's one of the K-1 camera issues since the SD cards are FAT32. Otherwise, as you said 4GB is a BIG file and you shouldn't run into issues with most other types of applications.
11-10-2018, 03:51 PM   #13
Site Supporter
Site Supporter
microlight's Avatar

Join Date: Sep 2011
Location: Hampshire, UK
Posts: 2,127
What was the laptop you had to send back, Mike? I’ve been using a Gigabyte P55 gaming laptop for a year or so (I’m a fanatical flight simulator user) hooked up to a Samsung HD 27-inch monitor and it’s been great so far.
11-10-2018, 04:23 PM   #14
Digitiser of Film
Loyal Site Supporter
BigMackCam's Avatar

Join Date: Mar 2010
Location: North East of England
Posts: 20,575
Original Poster
QuoteOriginally posted by jbinpg Quote
Hi, Mike. Firstly, I am sort of glad that you had to send your Chinese laptop back. It was giving you problems that should not have been there.

As for your file access question, there are usually two routes to access Windows files from linux: Samba/NTFS or NFS. Both methods have been mature for quite a while on the linux side and each have their advantages. Best you google for setups for these to see what fits your needs. Of course, we are relying on MS to not throw a wrench into their code somewhere but they have been cozying up to Ubuntu in particular the past few years so I doubt they would do this. Some say MS will eventually make a play to buy out Ubuntu's parent company Canonical in the not too distant future but that is speculation. Shuttleworth still has enough big bucks to fend off MS.

Jack
Jack - I tend to agree with you regarding the laptop I've returned. I chose that machine based on, it would seem, insufficient research, rendering the possibility of using any Linux distro seriously compromised for the time being. Even so, given that it was my choice to buy the machine rather than the supplier's to sell it, I'd have happily stuck with it under Windows 10 were it not for the fault and repair problems I experienced. Having paid a bit extra for fast build and delivery, sending it back for a second time was too much to bear, so a refund was justified. In a sense, I guess I've had a lucky escape

The downside is that I'm still having to use my old HP laptop which is gradually disintegrating around the screen hinges The upside is that, in researching a replacement, I'm much more clued up on what to look for and what to avoid. I like HP's active involvement with Ubuntu in testing and certifying compatibility, and that's one of the reasons I've decided to go with an HP mobile workstation. I know nothing's ever easy with a "perfect" Linux installation, but I've pretty good confidence I can get there with an HP workstation and their technical support guys.

Thanks for the info on file systems. I'll do some research to compare Samba/NTFS and NFS.

I hadn't heard the speculation around Microsoft acquiring Canonical That said, so long as it remains open source, I wouldn't have a huge problem with that... and if not, there are always other distros. In fact, I've been reading up on Manjaro recently, and that looks like an extremely interesting option. More work required for the user, it seems, but I like the rolling update aspect...

QuoteOriginally posted by K1N8 Quote
Sorry, I don't think there is any file system that can "reliably" be used across all operating systems, that supports modern features like journaling :/

I am thinking 10 years ahead here. That ext4 driver for win10 might break with a Windows update. FAT32 works everywhere, but it has too many restrictions, maximal file and disk size and things like that.

There is no simple choice that works for all :/
I have to admit my ignorance of journaling. But given that ext4 might break due to Windows update, and FAT32 is too restricted, what puts you off NTFS? I'm not looking for confirmation to use NTFS, but - I suppose - reasons why I shouldn't... Reasons why a different solution is better for the specific scenario I've mentioned

QuoteOriginally posted by Dartmoor Dave Quote
I've been dual booting Windows 7 and Ubuntu for a few years with the shared drive formatted to NTFS, and both operating systems have handled it okay. In fact, I've actually had more problems dual booting Windows 7 and Windows 10 causing a frequent need to run CHKDSK.

The main thing to watch out for is that Linux allows longer file names than Windows, which can be hassle. You have to then extract the 8 character file name from the Windows command line, and I once lost a file when that failed.
Thanks, Dave, that's useful info.

Re file names, as I understood it, long file names are supported by NTFS, Windows 10 and current Linux distros. If I remember correctly, NTFS has a limit of something of 255 characters for the entire path including file name, whereas Linux doesn't have that limit. Is that correct? Either way, whilst I do use quite lengthy names for my image files, they're not excessive, and my directory structure isn't complicated enough to result in large path descriptions - so I think I should be OK with NTFS on that basis...

QuoteOriginally posted by photoptimist Quote
IANAWU and IANALU (not a windows or linux user) but I've suffered through enough OS, storage, and filesystem nonsense to know where a few gotchas might lie.
Oooh, OK... You're MacOS, then?

QuoteOriginally posted by photoptimist Quote
First, I'm assuming and hoping that each OS accesses this disk while the other OS is completely shut down. That is, each OS gets to reliably unmount the HDD before the other OS is booted and mounts the HDD. If not, you might need some clever intermediate something to share the HDD.
Indeed, and a very good point.

I'm not running one operating system as a virtual machine under the other, or any other non-binary arrangement. Either I'll boot into Linux (which will be 90% of my usage) or into Windows. I have no intention of running the two simultaneously. Plus, if either operating system abended, I would go back in to fix things before attempting to boot into the other system.

QuoteOriginally posted by photoptimist Quote
Second, after you load your data the first time, run whatever disk/file system checking utilities you like on both OSes. Pay attention to any complaints/repairs required by one OS and then rerun the utility of the other OS to make sure they are going to fight over the file system structure.

Third, watch out for any OS updates or upgrades that "update" the file system or add features to it. These may break compatibility with the other OS.

Good luck and enjoy your new dual system!
Thanks - that's excellent advice
11-10-2018, 04:50 PM   #15
Digitiser of Film
Loyal Site Supporter
BigMackCam's Avatar

Join Date: Mar 2010
Location: North East of England
Posts: 20,575
Original Poster
QuoteOriginally posted by microlight Quote
What was the laptop you had to send back, Mike? I’ve been using a Gigabyte P55 gaming laptop for a year or so (I’m a fanatical flight simulator user) hooked up to a Samsung HD 27-inch monitor and it’s been great so far.
It was a custom build from PC Specialist Ltd. Actually, it was a very nice machine for the price I paid... i7-8750H, 16GB RAM, 512GB Intel 760p SSD, 2TB FireCuda SSHD, 17" 72% NTSC gamut screen, every port you could wish for - all for less than GBP £1,200. The issues I had upon delivery were a flickering screen when running on the power supply with battery fitted (no flickering when running just off the battery OR the power supply in isolation), and some missing lights on the multi-colour back-lit keyboard. The machine was collected and repaired - new LCD panel and keyboard - and returned to me, however the bezel around the screen had big gaps in places... enough to get four sheets of printer paper between the panel and bezel That's no good to me on a brand-new machine, so it was returned for refund as I didn't want to get into a potential back-and-forth loop to get a PC I felt was still "brand new" and fit for purpose.

Additionally, though - and nothing to do with my decision to ask for a refund - I found that it was poorly supported for use with Linux. As I found out shortly after buying the PC, it's essentially a Clevo N8xEJEK build, and it turns out that control of screen brightness, keyboard back-lighting and other aspects is highly dependent on Clevo's own proprietary control software ("Clevo Control Center") which only runs under Windows. I could live without keyboard back-lighting control, but screen brightness is of huge importance to me with profiling, photo editing and viewing. As such, I'd resigned myself (somewhat reluctantly) to running Win10 instead of Mint 18.3 or Mint 19 (my initial preference). That wouldn't have been the end of the world, as all the photo processing tools I use - digiKam, Darktable and GIMP - are available under Windows. As it happens, though, the problems with the supplied build quality resulted in me returning the unit anyway

In light of this, I've since found that Clevo builds are trouble for Linux if you want full control of all features (screen, keyboard, fan, etc.). So, in choosing an alternative, I've been looking very specifically for manufacturers and models that are either certified, or have significant evidence of success, with Ubuntu installations. In order of certification and support levels, that left me with Dell, closely followed by HP, and not too far after that, Lenovo.

Of the three, I personally like HP's offerings best - especially in terms of higher quality workstations. HP produces an extremely helpful and very honest matrix of PC feature compatibility with Ubuntu 16.04 and 18.04. Plus, the screen quality offerings in terms of colour gamut and brightness (in nits) for the most compatible models are very transparent to the potential buyer...

The PC I'm most likely to buy based on my current preferences is the HP ZBook 15 G5 with i7-8750H, Quadro P2000, 16GB RAM, UHD 4K Anti-glare DreamColor 600 nits screen, 512GB PCIe NVMe SSD and 2TB HDD. It's a big stretch for me financially (about twice what I paid for the PC Specialist machine), but it has more than enough room for my limited growth, and with a three year on-site warranty and service options beyond that, it should last me quite a while...

Last edited by BigMackCam; 11-11-2018 at 03:46 AM.
Reply

Bookmarks
  • Submit Thread to Facebook Facebook
  • Submit Thread to Twitter Twitter
  • Submit Thread to Digg Digg
Tags - Make this thread easier to find by adding keywords to it!
approach, format, hdd, linux, machine, ntfs, photography, photoshop, read/write access, software, ubuntu, windows
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads
Thread Thread Starter Forum Replies Last Post
Setting up Lightroom in Windows 10 - saving photos on different drive RussH Troubleshooting and Beginner Help 13 05-10-2018 11:02 AM
Accessing my Windows 10 folders from Ubuntu? BigMackCam General Talk 2 02-15-2017 09:24 AM
Read PEF of Pentax K-70 in Ubuntu? Umaxik Troubleshooting and Beginner Help 22 01-13-2017 02:43 AM
Firmware suggestion: Add setting to write video to both cards? PT1 Pentax K-3 & K-3 II 0 06-20-2014 04:17 PM
Time Exposure - Bulb setting - Read write times Vatis Pentax DSLR Discussion 7 09-05-2007 07:38 AM



All times are GMT -7. The time now is 03:10 AM. | See also: NikonForums.com, CanonForums.com part of our network of photo forums!
  • Red (Default)
  • Green
  • Gray
  • Dark
  • Dark Yellow
  • Dark Blue
  • Old Red
  • Old Green
  • Old Gray
  • Dial-Up Style
Hello! It's great to see you back on the forum! Have you considered joining the community?
register
Creating a FREE ACCOUNT takes under a minute, removes ads, and lets you post! [Dismiss]
Top