bash - Alert after long running shell script

When I have wait for slow commands (operations on large databases for example), I start doing something else and often don't notice for a long time that it has finished, wasting valuable time. I am looking for a way for the shell to alert me (by which I mean, run some custom command which I can determine) when a command which has been running for more than X seconds has finished and I got back the prompt.

I am using Bash, but solutions for other shells would be interesting as well.

Answer

This script, if put in your .bashrc, will execute any command you like whenever the prompt returns after a long time executing a command - customize the "long time" (after the -gt) in seconds, and within the curly braces on the same line you can execute whichever commands you like. You could even send yourself an email with the Host on which this occured and the last executed command and its status code.

beep_if_long_time_past() {
    LAST_COMMAND_DURATION=$(($(date +%s) - ${LAST_COMMAND_TIME}))
    [[ ${LAST_COMMAND_DURATION} -gt 60 ]] && { echo "Took long, didn't it?" ; notify-send "I'm done after ${LAST_COMMAND_DURATION} seconds!"; }
    export LAST_COMMAND_TIME=
}

export PROMPT_COMMAND=beep_if_long_time_past
trap '[ -z ${LAST_COMMAND_TIME} ] && export LAST_COMMAND_TIME=$(date +%s)' DEBUG

networking - Directly connect MacBook to Linux desktop via ethernet for fast SSH?

I am looking at building a Linux desktop—desktop only; no monitor support—and getting a MacBook Pro as my main computer for mobility. I would like—while I am at the office—to basically use my MacBook as my keyboard and main monitor, hook up another monitor or two to it and then SSH into my Linux console for my coding, etc.

Is it possible to directly connect my MacBook Pro to the Linux box via Ethernet? Or will I have to go through the building’s wired connection and then back into my Linux box?

Answer

Is it possible to directly connect my MacBook Pro to the Linux box via Ethernet?

Yes, you can directly connect systems via Ethernet. What you would do is on the Linux box you would assign a static IP address in /etc/network/interfaces something like this:

# The local hostmachine access interface.
auto eth1
iface eth1 inet static
address 192.168.99.10
netmask 255.255.255.0

Reboot your machine and now the eth1 physical port will have the IP address of 192.168.99.10.

Now on your Mac, hook up the Ethernet cable and set the network connection to the following IP address:

192.168.99.20

And set the “Subnet Mask” to 255.255.255.0 as well.

Then you will have a magical “two computer network” where the Linux machine has the IP address of 192.168.99.10 and your Mac’s Ethernet port has 192.168.99.20.

And to make your life even easier you can add the Linux equivalent of Bonjour broadcast networking by installing the Avahi daemon. On Ubuntu you would install it like this:

sudo aptitude install avahi-daemon avahi-utils

Then after it installs wait a second or two and on your Mac you will be able to reach the Linux box via it’s hostname. So let’s say your Linux box has a hostname of “LinuxDesktop” with Avahi installed it could be reached via the address of LinuxDesktop.local. And if you pinged LinuxDesktop.local it would return 192.168.99.10.

Now the big “gotcha” here is the actual network address scheme. I am using 192.168.99.x because typical LAN network addresses have the 192.168.x.x prefix octet. But in some cases it might be better for you to go with a 10.x.x.x address. What determines what is “better” is what address range your actual network outside of this Ethernet cable operates on. You basically do not want conflicts. Create a network address range for this Ethernet cable-based network based on an IP range your LAN—or even WAN—is not using.

But honestly, I am pretty sure addresses in the 192.168.99.x range will be fine.

worksheet function - How to simulate a full outer join in Excel?

Let's assume I have some data in Excel (and not in a real database). In one sheet, I have data, where one column functions as the ID, and I have made sure that the values in this column are unique. In another sheet, I also have some data, again with one column which can be taken as an ID, and it is also unique. If row N in Sheet 1 has some value, and row M in Sheet 2 has the same value, I am sure that row N and row M describe the same real-world object.

What I am asking: how can I get the equivalent of a full outer join without writing any macros? Formulas and all functions accessible through the ribbon are OK.

A small "play data" example:

Sheet 1:

Dostoyevski    Russia
Pushkin        Russia
Shelley        England
Flaubert       France
Hugo           France
Eichendorff    Germany
Byron          England
Zola           France

Sheet 2:

Shelley        Percy Bysshe
Eichendorff    Josef Freiherr Von
Flaubert       Gustave
Byron          Lord
Keller         Gottfried
Dostoyevski    Fyodor
Zola           Emile
Balzac         Honoré de

Desired output (sorting is not important):

Dostoyevski    Russia   Fyodor
Pushkin        Russia
Shelley        England  Percy Bysshe
Flaubert       France   Gustave
Hugo           France
Eichendorff    Germany  Josef Freiherr von
Byron          England  Lord
Zola           France   Emile
Keller                  Gottfried
Balzac                  Honoré de

---

To everybody who is horrified by this scenario: I know that this is The Wrong Way To Do It. If I have any choice, I would not use Excel for this. However, there are enough situations out there where a pragmatic solution is needed, stat, and a better (from IT point of view) solution cannot be applied.

Answer

First, copy/paste both key columns from both tables into a single, new sheet as a single column.

Use the "Remove Duplicates" get the single list of all your unique keys.

Then, add two columns (in this case), one for each of your data columns in each table. I recommend you use the format as table option too as it makes your formulas look much nicer. Using vlookup, use the following formula:

=IFERROR(VLOOKUP([@ID],Sheet4!A:B,2,FALSE),"")

Where Sheet4!A:B represents whatever the source table data table is for each respective value. The IFERROR prevents the ugly #N/A results which appear when vlookup is not successful and in this case return a blank cell.

This gives you your resulting table.

---

Sheet3:

Sheet4:

Result data:

Result formulas (Ctrl+~ will toggle this):

---

You can also do this with the built-in SQL query. It's... much less user friendly, but maybe will be a better use case. This will likely require you to have formatted your "source" data as tables.

1. Click on a cell in a new sheet


2. Go to Data --> From Other Sources --> From Microsoft Query


3. Select Excel Files* under the Databases tab and hit ok


4. Select your workbook


5. Select the following four fields:
   
   
   
   • 
   
   
   
   


6. Click "next" and "ok" at the nice 1990s formatted warning you see


7. Following these instructions create the first Left Outer Join. In my case I am using the "countries" table as the left source and the "names" as the right.
   
   
   
   • 
   
   
   • This only gives some of the rows (since you join on the ID)
   
   
   
   


8. 
   

   The "create a subtract join and then add it as union" part is more complicated..

   
   
   
   
   • Here is the subtract join configurations:
   
   
   • Copy this join's SQL from the SQL button:
   
   
   • 
     

     
     

     SELECT countries$.ID, countries$.Val1, names$.ID, names$.Val2 FROM {oj C:\Users\Username\Desktop\Book2.xlsx.countries$ countries$ LEFT OUTER JOIN C:\Users\Username\Desktop\Book2.xlsx.names$ names$ ON countries$.ID = names$.ID} WHERE (names$.ID Is Null)

     
     

     
     
   
   
   
   


9. 
   

   Go back to the first outer join you created. Manually edit the SQL and

   
   
   
   
   • add Union to the bottom
   
   
   • Add the above subtract join text to the bottom of the join
   
   
   
   


10. Hit the "Return Data" button immediately to the left of the SQL button
    
    
    
    • You may want to edit the SQL to only select the specific data you want at this point. I find it easier to hide columns in the result.
    
    
    
    


11. Place the Query somewhere and confirm it's location
    
    
    
    • 
    
    
    
    

Not for the faint of heart. But if you want a great chance to see some not-updated-as-long-as-you-might-have-been-alive parts of Office it's a great chance.

Windows 7 driver search woefully slow

Any thoughts on why the driver search (when installing a usb device) on windows 7 is woefully slow? Even when I click to disable trying to get a driver from windows update, it still takes 10min+ to either find a driver, or bail out.

Is there a way to speed this up, or alternately, disable windows trying to find drivers, so I can manually install them in device manager?

Answer

If its Windows 7, the time taken is incredibly long, especially if its set to use Windows update to scan for drivers on /every/ connect. That plus my flaky, flaky and slow Internet connection which resulted in timeouts means it took forever to detect. Asked Windows not to look at Windows update, and it was much better.

windows - Recover hard disk data after deleting partitions

When I reinstall Windows XP by mistake I deleted all five partitions. I did not format them yet though.

How can I recover the partitions? When I attach the hard disk to another PC it just says that the partition is unformated and asks "Would you like to format it?".

Answer

Use ralford's suggestion together with TestDisk :

TestDisk is a powerful free data recovery software! It was primarily designed to help recover lost partitions and/or make non-booting disks bootable again when these symptoms are caused by faulty software, certain types of viruses or human error (such as accidentally deleting a Partition Table). Partition table recovery using TestDisk is really easy.

Why is my motherboard's CPU temperature inconsistent with the individual core temperatures?

This image demonstrates what I'm talking about:

The motherboard's CPU temperature is in green. The individual cores' temperatures are in red.

I've tested the temperatures in three different scenarios:

• Idle: All cores <5% utilization. Core temperatures are around 70C, while CPU is around 55-60C. Frequency stays around 2600 Mhz.


• Gaming: Three of six cores around 50% utilization, remaining three <10%. Core temperatures gradually increase to 90C. CPU temperature gradually decreases to 40C. Frequency is gradually reduced to 1200 Mhz. Pictured above.


• Stressed: Using Intel's diagnostic tool to stress the CPU; all cores near 100% utilization. All cores are over 90C, while CPU is at 80C (I believe the tool limits itself if it goes above 80C). I can't monitor frequency during this.

Why are the two readings so different? Also, why does the motherboard's reading decrease while gaming, even though the Core readings are almost as high as they are during the stress test?

windows xp - Recurring crash: "An unhandled win32 exception occurred in mscorsvw.exe". How to diagnose/fix?

I recently got a new PC at work and had to reinstall development tools, etc. The PC runs Windows XP (blech), and I've got Visual Studio 2010 and .NET Frameworks 2.0, 3.5, and 4.0 installed, each with all current service packs and patches. Windows XP itself is also up-to-date (if one could say that :-)

One recurring problem I've noticed is the following dialog, which tends to pop up after the machine has been idle for a bit:

I suspect the crash is due to the .NET Framework performing Ngen compilation of system assemblies in the background, and crashing when it reaches one assembly in particular.

I've found another mention of this problem at the MSDN forums, and one of the suggested workarounds is to configure Windows XP's Data Execution Prevention feature to "Turn on DEP for essential Windows programs and services only". However, that's already the setting in effect on my PC.

How can I diagnose further? When I try to attach to the process, it's already gone.

Are there any other suggested or likely fixes?

---

UPDATE:

I found some more information on ngen here and here.

I ran the following at a Command Prompt: ngen executequeueditems .. this now lets me reliably reproduce the problem instead of waiting for the idle background ngen to execute.

So, when ngen.exe got to the following entry:

Compiling assembly Microsoft.SqlServer.Management.MultiServerConnection, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91 (CLR v2.0.50727) ...
WARNING: Cannot hardbind to mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 because dependency does not have a native image (check FusLogVw for reason)
Failed to generate native code for dependent image Microsoft.SqlServer.Management.MultiServerConnection, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91 because of the following error: The remote procedure call failed. (Exception from HRESULT: 0x800706BE)

... the result was the following error dialog:

Whereas, other assemblies that failed native image generation didn't actually cause a crash, just an error message.

So, the specific assembly this is failing on is: Microsoft.SqlServer.Management.MultiServerConnection.

What else can I do? I don't particularly care if it can be ngen'd or not at this point; I simply want to stop this annoying recurring error dialog from above. I already tried:

ngen uninstall Microsoft.SqlServer.Management.MultiServerConnection,

... but it says "ERROR: The specified assembly is not installed."

Is there a way to remove an assembly from the ngen queue, so ngen won't even try to generate a native image for it?

Answer

The nGen of the Sql Server assembly is failing because a previous attempt at nGening the .Net Framework 2.0 mscorlib assembly has failed. This is nothing to do with .Net 4 at all so I suggest you stop looking at it's ngen service (2 and 4 use their own services).

As for why that's failed - well there's not a huge amount around on the net about this. I did find this on MSDN forums though - there's something in there about confirming whether mscorlib has been correctly nGen'd - I'd check that out.

Interestingly Sql Server is mentioned on that thread too; although it's 2005 and I think version 10 assemblies you're talking about here are Sql 2008. Still, might provide something.

But ultimately if this were my machine I would:

• 
  

  Uninstall .Net 2.0 and 3.0 and 3.5 (if you're feeling brave uninstall 4.0 as well)

  
  


• 
  

  Uninstall whatever Sql Server component it is that contains the assembly that's failing the nGen.

  
  


• 
  

  Install the .Net 3.5 sp1 redist; making sure you have also got the hotfix that is referenced for further down the page as well.

  
  


• 
  

  Wait for all the nGening to complete before continuing.

  
  


• 
  

  If you uninstalled .Net 4 reinstall that now too.

  
  


• 
  

  Reinstall the Sql Server component removed in step 2.

  
  

Yes, I know, kinda a 'switch it off and on again' answer - but I think it's better than trying to sidestep the issue. If mscorlib 2.0 has not nGen'd correctly then tbh .Net 2.0 is not really usable in its current state. And since this is a SQL Server component that's trying to nGen - only an nGen'd mscorlib v2 will work with it (nGen'd mscorlib v4 will not).