Importing Data from CSV into CakePHP

This is a quick little method that lets you get some data from a CSV into a table.

It should work with very large CSV files as it only has to read one row at a time.

The Data

tmp/uploads/Post/posts.csv

Post.title, Post.description
My First Post, great posting with lots of information
Another Post, another interesting article

The Controller

controllers/posts_controller.php

<?php
class PostsController extends AppController {
	var $name = 'Posts';

	function import() {
		$messages = $this->Post->import('posts.csv');
		debug($messages);
	}
}
?>

The Model

models/post.php

<?php
class Post extends AppModel {
	var $name = 'Post';

	function import($filename) {
 		// to avoid having to tweak the contents of 
 		// $data you should use your db field name as the heading name 
		// eg: Post.id, Post.title, Post.description

		// set the filename to read CSV from
		$filename = TMP . 'uploads' . DS . 'Post' . DS . $filename;
		
		// open the file
 		$handle = fopen($filename, "r");
 		
 		// read the 1st row as headings
 		$header = fgetcsv($handle);
 		
		// create a message container
		$return = array(
			'messages' => array(),
			'errors' => array(),
		);

 		// read each data row in the file
 		while (($row = fgetcsv($handle)) !== FALSE) {
 			$i++;
 			$data = array();

 			// for each header field 
 			foreach ($header as $k=>$head) {
 				// get the data field from Model.field
 				if (strpos($head,'.')!==false) {
	 				$h = explode('.',$head);
	 				$data[$h[0]][$h[1]]=(isset($row[$k])) ? $row[$k] : '';
				}
 				// get the data field from field
				else {
	 				$data['Post'][$head]=(isset($row[$k])) ? $row[$k] : '';
				}
 			}

			// see if we have an id 			
 			$id = isset($data['Post']['id']) ? $data['Post']['id'] : 0;

			// we have an id, so we update
 			if ($id) {
 				// there is 2 options here, 
				 
				// option 1:
				// load the current row, and merge it with the new data
	 			//$this->recursive = -1;
	 			//$post = $this->read(null,$id);
	 			//$data['Post'] = array_merge($post['Post'],$data['Post']);
	 			
				// option 2:
	 			// set the model id
	 			$this->id = $id;
			}
			
			// or create a new record
			else {
	 			$this->create();
			}
 			
			// see what we have
			// debug($data);
			
 			// validate the row
			$this->set($data);
			if (!$this->validates()) {
				$this->_flash(,'warning');
				$return['errors'][] = __(sprintf('Post for Row %d failed to validate.',$i), true);
			}

 			// save the row
			if (!$error && !$this->save($data)) {
				$return['errors'][] = __(sprintf('Post for Row %d failed to save.',$i), true);
			}

 			// success message!
			if (!$error) {
				$return['messages'][] = __(sprintf('Post for Row %d was saved.',$i), true);
			}
 		}
 		
 		// close the file
 		fclose($handle);
 		
 		// return the messages
 		return $return;
 		
	}

}
?>

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

How about excel files? is there a better way?
There is a situation like this.

me: "Ok, you save your data as CSV file. then upload it in our system."

other: "CSV file?"

me: "Yes, CSV file."

other: "Im sorry i dont know CSV file."

OPTIONS 1. i could teach "other" how to do it. but i have
to teach "others" too. and the newcomers aswell.

OPTIONS 2. it would be better if they could just upload a
excel file. for their convenience.

what do you say?

brett's picture

You have 2 options:

1) Your user can File > Save As > CSV

2) convert XLS to CSV, there is an article here explaining a few ways:
http://bloke.org/php/converting-xls-to-csv-simple-eh-sort-of/

Althought a nice solution, it would be clever to encourage moving it to the model layer. Data handling on the controller side isn't very "pattern prone", it should only control your app flow, not it's data.

brett's picture

Thanks for the tip, I moved the bulk of the code into the model.

Keep your models fat and your controllers thin. =)

It looks like you simply copied and pasted the Controller code into the Model. That'll throw a bunch of errors, because the _flash method is not defined on the Model, an instead tries to do a Query. There must be another way to pass the error messages to the view via the controller...

brett's picture

yes, point taken. i have changed the model to return a list of errors and messages so the controller can handle them.

Just to point out you can save all of them in one go if you create the array like this, ignore my previous post

$data[$h[0]][$i][$h[1]]=(isset($row[$k])) ? $row[$k] : '';
$data['Post'][$i][$head]=(isset($row[$k])) ? $row[$k] : '';

brett's picture

Thanks for this tip!

Please note you also need to use saveAll() instead of save(). For more info see the cake cookbook saving your data

Hi,

you can use dbTube.org if you prefere "native" Excel sheet or the
customer can't provide CSV. This reduce the manual modification of
the file.

Markus

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <b> <i> <strong> <cite> <em> <code> <pre> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
  • You can enable syntax highlighting of source code with the following tags: <code>, <css>, <diff>, <drupal5>, <html>, <javascript>, <php>. The supported tag styles are: <foo>, [foo]. PHP source code can also be enclosed in <?php ... ?> or <% ... %>.

More information about formatting options

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.