Web transfers of extremely large directories can often fail for a variety of reasons. This guide will walk you through step by step instructions on how to get your folder of content uploaded to Pinata by transferring it directly from your local computer to us over the IPFS network. How cool is that?
1 - Pre-requirements:
The instructions provided in this guide must be run through what's known as "the command line".
To open this up in windows, check out: https://www.howtogeek.com/235101/10-ways-to-open-the-command-prompt-in-windows-10/
To open this up in macOS, check out: https://www.howtogeek.com/682770/how-to-open-the-terminal-on-a-mac/
Once you've opened up your command line application, you can begin typing in the commands provided in the installation documentation linked above.
2 - IPFS installation:
Once you have your command line application open, the first thing you're going to need to do is visit the ipfs official distribution page located here: https://docs.ipfs.io/install/command-line/#official-distributions. On this page, you'll need to scroll to your operating system and follow the instructions on how to install IPFS to your machine.
3 - Running IPFS
Once you've successfully installed IPFS, you're going to want to run it by typing
ipfs daemon into your command line.
You should see something like this appear:
If you see this, congratulations, you're running IPFS on your computer!
* Please note that sometimes you might see error messages related to networking in the command line window. As long as your IPFS daemon is still running, don't worry about these messages.
4 - Adding your content to IPFS
Once you have IPFS running, the next step is going to be to add your folder to your locally running IPFS node. To do this, you'll need to open up a new tab or window for your command line application. We need to run our future commands in a new window because we need to keep the ipfs daemon running in your original command line window.
On MacOS Windows you can do this by typing Ctrl + Win + T on your keyboard
On MacOS you can do this by typing ⌘ + n
In your new terminal window, you'll need to navigate to the location where your folder is being stored in your local filesystem.
You can do this by using the
To navigate into a folder location on your computer, type:
To navigate up a level in your computer, type:
To list the contents of the current folder you're located in, you can type:
Your target location is going to be the folder that contains the folder of files you want to upload to Pinata. So if your folder is named
exampleFolder and is stored in your
Documents folder on you'll want to make sure you're located in the
Documents folder, but not inside of the
exampleFolder folder. To confirm this, if you run the
ls command, you should see
exampleFolder listed in the results.
Once you're at the target location, you'll want to type the following command into your command line:
ipfs add --recursive --progress ./exampleFolder/
From here your folder should start being added to IPFS. If you have a relatively large folder, this may take some time. You'll know once this has finished when things stop being added and the upload reaches 100%. You should receive something that looks like this:
Notice the very bottom CID that we receive? The one that has the name of
exampleFolder next to it? That's what you'll want to grab.
5 - Make sure that your content is retreivable on the IFPS network.
5.1 - Before you do anything else, you'll want to check that your folder isn't too big for IPFS to transfer properly. To do this, the following command needs to be run:
ipfs block stat CID where CID is the root CID of your folder.
If the size of this exceeds 1000000, your content will not be discoverable!
Your solutions here are as follows:
1) wait for IPFS v0.11.0 to come out (which will include automatic directory sharding)
2) enable the experimental directory sharding feature on your local node as described here: https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#directory-sharding--hamt
Once you've enabled this, you'll need to restart your node before adding your folder again!
Once you've enabled this functionality and have re-added your content, you should receive a new CID. Run
ipfs block stat CID to double check that the result here is under 1000000.
5.2 - Now that you've added your content to your node and we know it fits within the IPFS folder limits, let's double check and make sure the public ipfs network can find it. To do this, we recommend visiting:
https://ipfs.io/api/v0/object/stat?arg=CID (replace CID with your folder's CID)
If this returns with some text telling you about your upload, and not an error, you're good to move on to the next step!
If you receive a timeout error, this means that your local node is likely behind a firewall has connectivity issues. To resolve these, we recommend checking out this guide here: https://docs.ipfs.io/how-to/nat-configuration/#nat-configuration
6 - Getting Pinata to retrieve your content
Now that you've added your content to your local IPFS node, it's time for Pinata to grab it!
go to https://app.pinata.cloud/pinmanager and click on the upload button and "CID":
In here, put in your CID and the name of it.
Once you've done that hit 'Search and Pin'
7 - Wait for success!
While Pinata is searching the network for your content, please be patient. Retrieval can take a long time if your folder is quite large. In addition, if your network connection is slow, this might also slow down retrieval times.
You'll know that the retrieval has finished by refreshing the pin manager every so often. Once your CID appears in the pin list, your content has been successfully pinned to Pinata!