I have a function that creates an image object from scratch using the PIL library. I want to use this image as an input to neural network without having to save the image and load it again
the following code creates an image with a rectangle in it
from PIL import Image, ImageDraw
img_size = 300
image = Image.new(mode='RGB', size=(img_size, img_size), color=0)
draw = ImageDraw.Draw(image)
draw.rectangle([(50, 50), (250, 250)], fill=255, outline=255, width=1)
image.show()
How would one go about using this object 'image' as in input to any neural network?
tensorflow and pytorch both are fine, I just need something that works
This is how you can pass PIL image to torch model
If image is your PIL <Image> class
Then
from torchvision import transforms
pil_to_tensor = transforms.ToTensor()(image).unsqueeze_(0)
output = model(pil_to_tensor ) #
Related
I have some .tif images and I'm reading them in as PIL image.
I know there is a ToPILimage transform
but I could not find a from_PILimage() akin to from_numpy()
as of right now I have this ugly looking thing:
img = torch.from_numpy(np.array(Image.open('path/image.tif')))
Could you show me a better way?
Thanks in advance!
Similar to torchvision.transforms.ToPILImage(), you can use torchvision.transforms.ToTensor() directly. Example from PyTorch docs
There's also the functional equivalent torchvision.functional.to_tensor().
img = Image.open('someimg.png')
import torchvision.transforms.functional as TF
TF.to_tensor(img)
from torchvision import transforms
transforms.ToTensor()(img)
Have 40 DICOM and 40 PNG images (data and their masks) for a Fully CNN that are loaded into my Google Drive and have been found by the notebook via the print(os.listdir(...)), as evidenced below in the first block of code where all the names of the 80 data in the above sets are listed.
Also have globbed all of the DICOM and PNG into img_path and mask_path, both with lengths of 40, in the second block of code that is below.
Now attempting to resize all of the images to 256 x 256 before inputting them into the U-net like architecture for segmentation. However, cannot load them via the nib.load() call, as it cannot work out the file type of the DCM and PNG files, even though for the latter it will not error but still produce an empty set of data like the last block of code yields.
Assuming that, once the first couple of lines inside the for loop in the third block of code are rectified, pre-processing should be completed and I can move onto the U-net implementation.
Have the current pydicom running in the Colab notebook and tried it in lieu of the nib.load() call, which produced the same error as the current one.
#import data as data
import pydicom
from PIL import Image
import numpy as np
import glob
import imageio
print(os.listdir("/content/drive/My Drive/Images"))
print(os.listdir("/content/drive/My Drive/Masks"))
pixel_data = []
images = glob.glob("/content/drive/My Drive/Images/IMG*.dcm");
for image in images:
dataset = pydicom.dcmread(image)
pixel_data.append(dataset.pixel_array)
#print(len(images))
#print(pixel_data)
pixel_data1 = [] ----------------> this section is the trouble area <-------
masks = glob.glob("content/drive/My Drive/Masks/IMG*.png");
for mask in masks:
dataset1 = imageio.imread(mask)
pixel_data1.append(dataset1.pixel_array)
print(len(masks))
print(pixel_data1)
['IMG-0004-00040.dcm', 'IMG-0002-00018.dcm', 'IMG-0046-00034.dcm', 'IMG-0043-00014.dcm', 'IMG-0064-00016.dcm',....]
['IMG-0004-00040.png', 'IMG-0002-00018.png', 'IMG-0046-00034.png', 'IMG-0043-00014.png', 'IMG-0064-00016.png',....]
0 ----------------> outputs of trouble area <--------------
[]
import glob
img_path = glob.glob("/content/drive/My Drive/Images/IMG*.dcm")
mask_path = glob.glob("/content/drive/My Drive/Masks/IMG*.png")
print(len(img_path))
print(len(mask_path))
40
40
images=[]
a=[]
for a in pixel_data:
a=resize(a,(a.shape[0],256,256))
a=a[:,:,:]
for j in range(a.shape[0]):
images.append((a[j,:,:]))
No output, this section works fine.
images=np.asarray(images)
print(len(images))
10880
masks=[] -------------------> the other trouble area <-------
b=[]
for b in masks:
b=resize(b,(b.shape[0],256,256))
b=b[:,:,:]
for j in range(b.shape[0]):
masks.append((b[j,:,:]))
No output, trying to solve the problem of how to fix this section.
masks=np.asarray(masks) ------------> fix the above section and this
print(len(masks)) should have no issues
[]
You are trying to load the DICOM files again using nib.load, which does not work, as you already found out:
for name in img_path:
a=nib.load(name) # does not work with DICOM files
a=a.get_data()
a=resize(a,(a.shape[0],256,256))
You already have the data from the DICOM files in the pixel_data list, so you should use these:
for a in pixel_data:
a=resize(a,(a.shape[0],256,256)) # or something similar, depending on the shape of pixel_data
...
Your last loop for mask in masks: is never executed because two lines about it you set masks = [].
It looks like it should to be for mask in mask_path:. mask_path is the list of mask file names.
So I've been using Tensorflow's tutorials for neural networks. I completed the "basic classification" that is essentially just MNIST and have been working on making my own custom variation as a little thought experiment. Everything is pretty self explanatory except putting the datasets into a usable form as the tutorial uses a premade dataset and looks like it cuts some corners. All I would like to know is how to put a colored photo into a usable piece of data. I assume that will just be a 1D array. As a side question, will a neural network lose any effectiveness if a 2d photo is stored in a 1d array if its not a CNN.
Datasets included in Keras are premade and usually preprocessed so that beginner could easily try a hand on them. For using your own images, like for a cat-dog image classification problem, you can place the images in two separate directories, for example,
in images/cats and images/dogs.
Now, we parse each and every image in these directories,
import os
from PIL import Image
master_dir = 'images'
img_dirs = os.listdir( master_dir )
for img_dir in img_dirs:
img_names = os.listdir( os.path.join( master_dir , img_dir ) )
for name in img_names:
img_path = os.path.join( master_dir , img_dir , name )
image = Image.open( img_path ).resize( ( 64 , 64 ) ).convert( 'L' )
# Store this image in an array with its corresponding label
Here. the image will be an array of shape (64, 64 ) which indicates that the image is grayscale. Besides .convert( 'L' ) in the code, we can use .convert( 'RGB' ) to have an image of shape (64,64,3) RGB image.
Now,
Collect all the images and labels in a Python list.
Convert the lists to NumPy arrays.
Store the NumPy arrays in a .npy file using the np.save() method.
In the file which trains the model, load these files using np.load() method and feed them to the model.
I'm working on some large histological images using Vips image library. Together with the image I have an array with coordinates. I want to make a binary mask which masks out the part of the image within the polygon created by the coordinates. I first tried to do this using vips draw function, but this is very inefficiently and takes forever (in my real code the images are about 100000 x 100000px and the array of polygons are very large).
I then tried creating the binary mask using PIL, and this works great. My problem is to convert the PIL image into an vips image. They both have to be vips images to be able to use the multiply-command. I also want to write and read from memory, as I believe this is faster than writing to disk.
In the im_PIL.save(memory_area,'TIFF') command I have to specify and image format, but since I'm creating a new image, I'm not sure what to put here.
The Vips.Image.new_from_memory(..) command returns: TypeError: constructor returned NULL
from gi.overrides import Vips
from PIL import Image, ImageDraw
import io
# Load the image into a Vips-image
im_vips = Vips.Image.new_from_file('images/image.tif')
# Coordinates for my mask
polygon_array = [(368, 116), (247, 174), (329, 222), (475, 129), (368, 116)]
# Making a new PIL image of only 1's
im_PIL = Image.new('L', (im_vips.width, im_vips.height), 1)
# Draw polygon to the PIL image filling the polygon area with 0's
ImageDraw.Draw(im_PIL).polygon(polygon_array, outline=1, fill=0)
# Write the PIL image to memory ??
memory_area = io.BytesIO()
im_PIL.save(memory_area,'TIFF')
memory_area.seek(0)
# Read the PIL image from memory into a Vips-image
im_mask_from_memory = Vips.Image.new_from_memory(memory_area.getvalue(), im_vips.width, im_vips.height, im_vips.bands, im_vips.format)
# Close the memory buffer ?
memory_area.close()
# Apply the mask with the image
im_finished = im_vips.multiply(im_mask_from_memory)
# Save image
im_finished.tiffsave('mask.tif')
You are saving from PIL in TIFF format, but then using the vips new_from_memory constructor, which is expecting a simple C array of pixel values.
The easiest fix is to use new_from_buffer instead, which will load an image in some format, sniffing the format from the string. Change the middle part of your program like this:
# Write the PIL image to memory in TIFF format
memory_area = io.BytesIO()
im_PIL.save(memory_area,'TIFF')
image_str = memory_area.getvalue()
# Read the PIL image from memory into a Vips-image
im_mask_from_memory = Vips.Image.new_from_buffer(image_str, "")
And it should work.
The vips multiply operation on two 8-bit uchar images will make a 16-bit uchar image, which will look very dark, since the numeric range will be 0 - 255. You could either cast it back to uchar again (append .cast("uchar") to the multiply line) before saving, or use 255 instead of 1 for your PIL mask.
You can also move the image from PIL to VIPS as a simple array of bytes. It might be slightly faster.
You're right, the draw operations in vips don't work well with very large images in Python. It's not hard to write a thing in vips to make a mask image of any size from a set of points (just combine lots of && and < with the usual winding rule), but using PIL is certainly simpler.
You could also consider having your poly mask as an SVG image. libvips can load very large SVG images efficiently (it renders sections on demand), so you just magnify it up to whatever size you need for your raster images.
How do I do the above ? This is my code but it doesn't work nothing is displayed
from PIL import Image
import glob
image_list = []
for filename in glob.glob('<my directory>.pgm'):
im=Image.open(filename)
image_list.append(im)
import matplotlib.pyplot as plt
for i in range(10):
plt.figure()
plt.imshow(image_list[i])
I would like it to be displayed in the cell
If you're interested in a much simpler and faster way of displaying images I recommend IPyPlot package:
import ipyplot
ipyplot.plot_images(images_list, max_images=20, img_width=150)
It's capable of displaying hundreds of images in a grid-like format within just 60-70ms
You would get a plot similar to this:
In your case, you should add %matplotlib inlinebefore your code and make sure that after plt.imshow(images_list) you add plt.show() as well so matplotlib renders your images.