Robert Basic's blog

Posts tagged 'linux'

Find duplicate lines from a CSV

by Robert Basic on March 21, 2016.

Finding duplicate lines from a CSV file is something I have to do from time to time, yet not on a regular enough basis to remember it all. Plus, I’m trying to blog more often.

cut -d, -f1 file.csv | tr -d '"' | sort | uniq -dc

cut to split the lines at the commas and select the first field, then tr to delete any double quotes that encloses the field, then sort and finally with uniq to show only the duplicated lines and to prefix every line with the count of occurrences.

Tags: linux, shell.
Categories: Development.

Need help on your PHP projects? Let's talk!

When a package update goes wrong

by Robert Basic on February 06, 2013.
Heads-up! You're reading an old post and the information in it is quite probably outdated.

I am running Fedora 17 on my laptop, and yesterday there were some packages to update. Nothing unusual, updates on Fedora are quite frequent and, up until yesterday, there was not a single problem I remember with any update. And it was a small update, four packages in total. What could possibly go wrong, right?

After a reboot an odd thing happened. My laptop did not automatically connect to the wifi. Huh. So I clicked on the little network manager icon to pick the wifi. The list of scanned wifis was empty. Not a single network on the list. Then I did all the usual things one could do in a situation like this - turn on/off the wifi via the network manager, turn on/off the wifi via the keyboard shortcut, reboot the laptop a couple of times. Still nothing. At this point I started panicking.

I started suspecting the update. But I have no idea what packages were actually updated, what those packages do, or how I could see what packages were updated. After a bit of a googling from my phone, the answer was yum history. Apparently, this command will list recent changes to the system done by yum. Each change has a Transaction ID. More information about each change can be get with yum history info TID, where TID would be the transaction ID of the change you’re after. That will nicely list when did the change occur, which user made the changes, and what packages were affected in what way by that change.

One package caught my eye, called crda. It was updated from version 1.1.2 to 1.1.3. Google told me that crda has something to do with wireless. So, that was probably the culprit of my broken wireless. I started searching for possible bugs for this, or maybe even a workaround or a fix, but to no avail. I am not really good at debugging things like this, so I started looking for a way to somehow revert the update to this package and hopefully fix my wireless problems.

Yet again Google was involved and yet again yum history came to rescue. Apparently, besides tracking changes to the packages, it can also undo these changes: yum history undo TID, where TID is the ID of the transaction you want undone. It will try to undo changes to all packages, and in my case, it failed to undo some (hi Java!), but the crda package was reverted back to version 1.1.2 and after another reboot, the wireless was up and running once again, like nothing happened.

Yey for yum!

I have submitted a bug to Fedora’s bugzilla to hopefully figure out what went wrong and help the developers find a fix for this.

Tags: fedora, history, linux, packages, undo, yum.
Categories: Development, Software.

Need help on your PHP projects? Let's talk!

Moblin, Linux for netbooks

by Robert Basic on May 21, 2009.
Heads-up! You're reading an old post and the information in it is quite probably outdated.

Moblin got me curios and I wanted to test it out:

Moblin is an open source project focused on building a Linux-based platform optimized for the next generation of mobile devices including Netbooks, Mobile Internet Devices, and In-vehicle infotainment systems.

Cause I don’t own (yet!) a netbook, I installed it under VirtualBox (VB from now on). The image is 666 MB big and it comes not in an .iso, but in a .img format. But, VB, a really awesome software, had no troubles booting from it. As with the majority of Linux distros nowadays, Moblin image is also a Live CD, which means you can run it, without installing it.

The preinstall process is made up from 6-7 steps: choosing the language, the keyboard layout, the timezone and, of course, the partitioning. Basically, it’s just another boring “Next-Next” process. The installation itself took around 6 minutes to finish. When it’s done, it asks for a username and a password.

The first boot went pretty quickly, considering that booting under VB takes longer than booting under regular installations. The thing about VB is that it needs, the so called “Guest Additions” installed on the guest machine, so that the guest machine can be used normally. In this case, I failed to install it: Moblin comes with one version of the Linux kernel and the additions are for another version of the kernel. This prevented me in my quest to test Moblin fully. Anyway, I’ve managed to take a few screenshots of it, all are uploaded to my Picasa profile.

There was one thing that was strange. It has a “Status panel”, from which you can update your profiles on social networks. A really useful stuff. I just opened it up and updated my Twitter profile. Almost. I wasn’t logged in to Twitter from it and Moblin didn’t say a word about it. It just happily said that my status is updated. Once I found the “Web services” panel I logged in and this time I was really updating my Twitter stream.

I really was hoping to test it normally and write a detailed review of it, but this guest additions thingy thought otherwise. Moblin is a great distro, even in this beta stage I believe it’s useful. What do you think? Did you test it already, saw it in action?

One thing’s for sure: when I’ll get myself a netbook, it’ll run on Moblin.

Cheers!

P.S.: Check out the Moblin intro, too!

Tags: about, introduction, linux, moblin, netbook, open source, random.
Categories: Blablabla, Free time, Software.

Need help on your PHP projects? Let's talk!

Trac on Ubuntu

by Robert Basic on January 27, 2009.
Heads-up! You're reading an old post and the information in it is quite probably outdated.

Today I was messing around with Trac, installing it and doing some basic configuration. While my dev machine gets updated, I want to share my process of installing Trac.

What is Trac?

As said on the Trac homepage:

Trac is an enhanced wiki and issue tracking system for software development projects.

It’s free, it’s open source, it comes under the BSD license and it’s really awesome. You can write a wiki with it, have a ticket system, connect it with SVN, so you can browse the sources from the browser and see all the commit messages, when was something changed, added… It can support one project, it can support multiple projects. It can be viewable/editable by anyone, or you can close it down for your little team…

Trac is big. It has lots of plug-ins, so you can extend and customize your Trac. I haven’t played with them yet, but as soon as I will, you’ll get notified ;)

It’s written in Python. It can run on it’s own server, or it can run under Apache (where there are also several options). It can use SQlite, PostrgeSQL or MySQL databases. Currently it can connect only to SVN.

I’ll show you how to setup a basic Trac 0.11-dot-something-dot-something. It will run under Apache with mod_wsgi, use a SQlite database, connect to the SVN repository and require user authentication.

Installing Trac

Before anything, I want to say that my machine where I installed Trac has LAMP and SVN configured like this. So, this post is kinda the next part of that post.

First, I installed a Python tool, called Easy Install. It’s here to make our installation process easier. Lovely. Go to http://pypi.python.org/pypi/setuptools/, scroll down to the downloads section and choose a Python egg to download (match it to your currently installed Python version — I have Python 2.5 so I downloaded “setuptools-0.6c9-py2.5.egg”).

Fire up a console and type:

sudo sh setuptools-0.6c9-py2.5.egg

Of course, you need to match this to your own setuptools file.

Next, type:

sudo easy_install Trac

EasyInstall will now locate Trac and it’s dependencies, download and install them.

Download the mod_wsgi:

sudo apt-get install libapache2-mod-wsgi

It will install and enable mod_wsgi. And, in my case, it only tried to restart Apache, but for an unknown reason it fails to do so. If that happens, just do a quick:

sudo /etc/init.d/apache2 restart

If you want Subversion with your Trac, you’ll need the python-subversion package:

sudo apt-get install python-subversion

If you have it already, it’ll just skip it. If you want SVN, but you don’t have this package, later on it will show an error message like: Unsupported version control system “svn”.

Now to make a folder for Trac, where it will keep all the Trac projects and stuff.

sudo mkdir /var/trac /var/trac/sites /var/trac/eggs /var/trac/apache
sudo chown -R www-data /var/trac

Under /var/trac/sites will be the files for Trac projects. The /var/trac/eggs folder will be used as a cache folder for Python eggs. /var/trac/apache will hold a wsgi script file.

The wsgi script is actually a Python script, but with the .wsgi extension, used by mod_wsgi. With this script, Trac will be able to run as a WSGI application.
File: /var/trac/apache/trac.wsgi

import sys
sys.stdout = sys.stderr

import os
os.environ['TRAC_ENV_PARENT_DIR'] = '/var/trac/sites'
os.environ['PYTHON_EGG_CACHE'] = '/var/trac/eggs'

import trac.web.main

application = trac.web.main.dispatch_request

With this kind of script, one single Trac installation will be able to manage multiple projects (you can see here some other scripts).

Configure Apache, add this to your httpd.conf file:

WSGIScriptAlias /trac /var/trac/apache/trac.wsgi

<Directory /var/trac/apache>
    WSGIApplicationGroup %{GLOBAL}
    Order deny,allow
    Allow from all
</Directory>

Restart Apache:

sudo /etc/init.d/apache2 restart

If you go to http://localhost/trac/ in your browser, you should see an empty list of Available Projects. It’s empty, cause we haven’t added any project yet.

Now, let’s asume that we have a project called “testProject” with it’s source located in /var/www/testProject and a SVN repo located in /var/svn/repos/testProject. I’ll show how to add that project to Trac.

In console type:

sudo trac-admin /var/trac/sites/testProject initenv

Note that you need to provide the full path to /var/trac/sites, cause it will create a Trac project in the current folder you’re in.

It will ask you now a few things:

  • Project Name — the name of the project, e.g. “Trac testing project”
  • Database connection string — leave it empty, and it will use SQlite
  • Repository type — leave it empty, and it will use SVN
  • Path to repository — path to the project repo, e.g. /var/svn/repos/testProject

It will start to print out a bunch of lines, about what is it doing. In the end you’ll get a message like “Project environment for ‘testProject’ created.” and a few more lines. One more thing. We need to add the whole project to www-data user, so it can manage the files:

sudo chown -R www-data /var/trac/sites/testProject

If you direct your browser again to http://localhost/trac/, you will now see a link for the testProject. Click it. There, a fully working basic Trac environment for your project. A wiki, a ticket/bug tracking system, a repo browser in only a few minutes. How cool is that? Very.

This Trac environment can now be accessible by everyone. If we do not want that, we need to add this to the httdp.conf file:

<Location /trac>
    AuthType Basic
    AuthName "Trac login"
    AuthUserFile /var/trac/.htpasswd
    Require valid-user
</Location>

Create the .htpasswd file:

sudo htpasswd -bcm /var/trac/.htpasswd your_username your_password

All set. You’ll now have to login to Trac to be able to work on it. As I’m the big boss on my localhost, I gave myself some super-power privileges for Trac: TRAC_ADMIN. It’s like root on *NIX.

sudo trac-admin /var/trac/sites/testProject permission add robert TRAC_ADMIN

Read more about privileges.

That would be it. With this kind of setup, for now, it’s working perfectly for me. For Trac that’s available from the whole Internet, more security measures are needed, but this is only on localhost, so this is enough for me.

Comments, thoughts, ideas?

Happy hacking!

Tags: apache, example, lamp, linux, setup, svn, trac, ubuntu.
Categories: Development, Software.

Need help on your PHP projects? Let's talk!

Ubuntu as a dev machine

by Robert Basic on October 15, 2008.
Heads-up! You're reading an old post and the information in it is quite probably outdated.

This post is more of a note to myself, ‘cause I keep forgetting all these Linux commands, and spend hours setting up stuff right…

I’m installing Ubuntu 8.04 on VirtualBox, with windows xp as the host machine. I must do it this way, because my wireless card is having some problems with Linux, something with the drivers. The possible solution includes kernel compiling — thanks, but no thanks.

Anyway… The installation itself is no trouble, so I’ll skip that. I always keep the apt-cache from previous installations, sparing hours of updating the system… On the host I have a folder that I share between the host OS and the client OS and first I need to reach that folder, to get from it the apt-cache.

First, need to install the Guest Additions. In Virtualbox go to Devices —> Install Guest Additions. In the console run:

sudo /media/cdrom/VBoxLinuxAdditions.run

After it’s finished, we need to mount the shared folder:

sudo mount -t vboxsf name_of_the_sharing_folder /path/to/mount_point

Now, for me, this command shows some error. Here’s what I have to do:

sudo modprobe vboxfs
sudo mount -t vboxsf name_of_the_sharing_folder /path/to/mount_point

Something with some modules not being loaded into the kernel, not bothered with it really… Now I can copy the apt-cache to where it needs to be:

sudo cp -r /path/to/mount_point/apt-cache /var/cache/apt/archives

Now do the system update. If the system update includes a kernel update, you’ll have to install Guest Additions once more…

Next installing the LAMP:

sudo apt-get install apache2
sudo apt-get install php5 libapache2-mod-php5
sudo /etc/init.d/apache2 restart
sudo apt-get install mysql-server
sudo apt-get install libapache2-mod-auth-mysql php5-mysql phpmyadmin
sudo /etc/init.d/apache2 restart
sudo a2enmod rewrite
sudo /etc/init.d/apache2 restart

That should do it. But hey! mod_rewrite still doesn’t work!

sudo gvim /etc/apache2/sites-available/default

And change AllowOverride None to AllowOverride All.

There. I have a basic LAMP on Ubuntu under VirtualBox. I made a few snapshots of the VirtualBox image, in case I trash it (which probably will happen soon), so I don’t need to reinstall over again.

Now, I’m of to setup SVN…

Tags: apache, lamp, linux, mysql, php, setup, ubuntu, virtualbox.
Categories: Development, Software.

Need help on your PHP projects? Let's talk!